Skip to contentSkip to conversation

mptcp: just another recvmsg refactor

RFC PATCH In review Paolo Abeni 2 participants 8 messages Oct 13, 2019 – Oct 15, 2019

Subsystems (2)
lore
Share & tools
lore

Paolo Abeni

author
#0 lore raw

cover mptcp: just another recvmsg refactor

This start constructing the instrucutre to support pulling data from multiple subflow. At the current stage uses a very simply approach, dropping any out-of-order data and reading from any subflow with valid data.

The above should work quite nicely in active backup scenario and possibly even with multiple subflows sending data concurrently - with a low bwidth and a huge number of retransmissions.

This is a not_so_early draft, shared as it may pave some ground and/or partially overlap with DATA_FIN support effort

Note that this rewrite completely the recvmsg() main loop to mirror more closely the TCP recvmsg() main loop, likely fixing some bug in respect to signal and error condition handling.

Paolo Abeni (3):
mptcp: move some helper into the header file
mptcp: flush duplicate data at data_ready() time
mptcp: recvmsg() can drain data from multiple subflows

 net/mptcp/protocol.c | 360 +++++++++++++------------------------------
 net/mptcp/protocol.h |  31 +++-
 net/mptcp/subflow.c  | 270 +++++++++++++++++++++++++++++++-
 3 files changed, 403 insertions(+), 258 deletions(-)
Signature · 1 line
2.21.0
author
#1 lore raw

RFC PATCH 1/3 mptcp: move some helper into the header file

and add scope prefix, where needed. This is apreparatory work to make the next patch smaller.

The split is very arbitrary, likely a different one will be needed to simplify merging the patches.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/protocol.c | 25 +++++--------------------
 net/mptcp/protocol.h | 21 +++++++++++++++++++++
 2 files changed, 26 insertions(+), 20 deletions(-)
net/mptcp/protocol.c +5 -20
index cf4d68602edb..45cea5edf9a5 100644
@@ -387,10 +387,6 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	return ret;
 }
 
-struct mptcp_read_arg {
-	struct msghdr *msg;
-};
-
 static u64 expand_seq(u64 old_seq, u16 old_data_len, u64 seq)
 {
 	if ((u32)seq == (u32)old_seq)
@@ -400,19 +396,7 @@ static u64 expand_seq(u64 old_seq, u16 old_data_len, u64 seq)
 	return seq | ((old_seq + old_data_len + 1) & GENMASK_ULL(63, 32));
 }
 
-static u64 get_map_offset(struct mptcp_subflow_context *subflow)
-{
-	return tcp_sk(mptcp_subflow_tcp_socket(subflow)->sk)->copied_seq -
-		      subflow->ssn_offset -
-		      subflow->map_subflow_seq;
-}
-
-static u64 get_mapped_dsn(struct mptcp_subflow_context *subflow)
-{
-	return subflow->map_seq + get_map_offset(subflow);
-}
-
-static int mptcp_read_actor(read_descriptor_t *desc, struct sk_buff *skb,
+int mptcp_read_actor(read_descriptor_t *desc, struct sk_buff *skb,
 			    unsigned int offset, size_t len)
 {
 	struct mptcp_read_arg *arg = desc->arg.data;
@@ -666,8 +650,9 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 			break;
 		}
 
-		ack_seq = get_mapped_dsn(subflow);
-		map_remaining = subflow->map_data_len - get_map_offset(subflow);
+		ack_seq = mptcp_subflow_get_mapped_dsn(subflow);
+		map_remaining = subflow->map_data_len -
+				mptcp_subflow_get_map_offset(subflow);
 
 		if (before64(ack_seq, old_ack)) {
 			/* Mapping covers data already received, discard data
@@ -689,7 +674,7 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 			break;
 
 		/* Refresh current MPTCP sequence number based on subflow seq */
-		ack_seq = get_mapped_dsn(subflow);
+		ack_seq = mptcp_subflow_get_mapped_dsn(subflow);
 
 		if (before64(old_ack, ack_seq))
 			msk->ack_seq = ack_seq;
net/mptcp/protocol.h +21 -0
index 3c66f69a734d..98080d9f72fd 100644
@@ -245,6 +245,20 @@ mptcp_subflow_tcp_socket(const struct mptcp_subflow_context *subflow)
 	return subflow->tcp_sock;
 }
 
+static inline u64
+mptcp_subflow_get_map_offset(struct mptcp_subflow_context *subflow)
+{
+	return tcp_sk(mptcp_subflow_tcp_socket(subflow)->sk)->copied_seq -
+		      subflow->ssn_offset -
+		      subflow->map_subflow_seq;
+}
+
+static inline u64
+mptcp_subflow_get_mapped_dsn(struct mptcp_subflow_context *subflow)
+{
+	return subflow->map_seq + mptcp_subflow_get_map_offset(subflow);
+}
+
 int mptcp_is_enabled(struct net *net);
 
 void mptcp_subflow_init(void);
@@ -256,6 +270,13 @@ extern const struct inet_connection_sock_af_ops ipv4_specific;
 
 void mptcp_proto_init(void);
 
+struct mptcp_read_arg {
+	struct msghdr *msg;
+};
+
+int mptcp_read_actor(read_descriptor_t *desc, struct sk_buff *skb,
+		     unsigned int offset, size_t len);
+
 void mptcp_get_options(const struct sk_buff *skb,
 		       struct tcp_options_received *opt_rx);
 
Signature · 1 line
2.21.0
author
#2 lore raw

RFC PATCH 2/3 mptcp: flush duplicate data at data_ready() time

In reply to Paolo Abeni

Move subflow mapping related function to subflow.c - I think they belong there.

Refactor slightly get_mapping() to additionally validate the current mapping, moving some code from mptcp_recvmsg().

mptcp_data_ready() drops any skb which does not match exactly the current MPTCP seq number.

Old seqs are likely spurious retransmit we really want to ignore "future" seq may really happen in case of active backup switch-over, but we should converge to the current seq via 'correct' MPTCP-level retransmissions.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/protocol.c | 187 +-----------------------------
 net/mptcp/protocol.h |  10 +-
 net/mptcp/subflow.c  | 270 ++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 279 insertions(+), 188 deletions(-)
net/mptcp/protocol.c +6 -181
index 45cea5edf9a5..da2d749d69f4 100644
@@ -387,17 +387,8 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	return ret;
 }
 
-static u64 expand_seq(u64 old_seq, u16 old_data_len, u64 seq)
-{
-	if ((u32)seq == (u32)old_seq)
-		return old_seq;
-
-	/* Assume map covers data not mapped yet. */
-	return seq | ((old_seq + old_data_len + 1) & GENMASK_ULL(63, 32));
-}
-
 int mptcp_read_actor(read_descriptor_t *desc, struct sk_buff *skb,
-			    unsigned int offset, size_t len)
+		     unsigned int offset, size_t len)
 {
 	struct mptcp_read_arg *arg = desc->arg.data;
 	size_t copy_len;
@@ -425,88 +416,6 @@ int mptcp_read_actor(read_descriptor_t *desc, struct sk_buff *skb,
 	return copy_len;
 }
 
-enum mapping_status {
-	MAPPING_ADDED,
-	MAPPING_MISSING,
-	MAPPING_EMPTY,
-	MAPPING_DATA_FIN
-};
-
-static enum mapping_status mptcp_get_mapping(struct sock *ssk)
-{
-	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
-	struct mptcp_ext *mpext;
-	enum mapping_status ret;
-	struct sk_buff *skb;
-	u64 map_seq;
-
-	skb = skb_peek(&ssk->sk_receive_queue);
-	if (!skb) {
-		pr_debug("Empty queue");
-		return MAPPING_EMPTY;
-	}
-
-	mpext = mptcp_get_ext(skb);
-
-	if (!mpext) {
-		/* This is expected for non-DSS data packets */
-		return MAPPING_MISSING;
-	}
-
-	if (!mpext->use_map) {
-		ret = MAPPING_MISSING;
-		goto del_out;
-	}
-
-	pr_debug("seq=%llu is64=%d ssn=%u data_len=%u", mpext->data_seq,
-		 mpext->dsn64, mpext->subflow_seq, mpext->data_len);
-
-	if (mpext->data_len == 0) {
-		pr_err("Infinite mapping not handled");
-		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_INFINITEMAPRX);
-		ret = MAPPING_MISSING;
-		goto del_out;
-	} else if (mpext->subflow_seq == 0 &&
-		   mpext->data_fin == 1) {
-		pr_debug("DATA_FIN with no payload");
-		ret = MAPPING_DATA_FIN;
-		goto del_out;
-	}
-
-	if (!mpext->dsn64) {
-		map_seq = expand_seq(subflow->map_seq, subflow->map_data_len,
-				     mpext->data_seq);
-		pr_debug("expanded seq=%llu", subflow->map_seq);
-	} else {
-		map_seq = mpext->data_seq;
-	}
-
-	if (subflow->map_valid) {
-		/* due to GSO/TSO we can receive the same mapping multiple
-		 * times, before it's expiration.
-		 */
-		if (subflow->map_seq != map_seq ||
-		    subflow->map_subflow_seq != mpext->subflow_seq ||
-		    subflow->map_data_len != mpext->data_len) {
-			MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_DSSNOMATCH);
-			pr_warn("Replaced mapping before it was done");
-		}
-	}
-
-	subflow->map_seq = map_seq;
-	subflow->map_subflow_seq = mpext->subflow_seq;
-	subflow->map_data_len = mpext->data_len;
-	subflow->map_valid = 1;
-	ret = MAPPING_ADDED;
-	pr_debug("new map seq=%llu subflow_seq=%u data_len=%u",
-		 subflow->map_seq, subflow->map_subflow_seq,
-		 subflow->map_data_len);
-
-del_out:
-	__skb_ext_del(skb, SKB_EXT_MPTCP);
-	return ret;
-}
-
 static void mptcp_wait_data(struct sock *sk, long *timeo)
 {
 	DEFINE_WAIT_FUNC(wait, woken_wake_function);
@@ -532,12 +441,6 @@ static void mptcp_wait_data(struct sock *sk, long *timeo)
 	remove_wait_queue(sk_sleep(sk), &wait);
 }
 
-static void warn_bad_map(struct mptcp_subflow_context *subflow, u32 ssn)
-{
-	WARN_ONCE(1, "Bad mapping: ssn=%d map_seq=%d map_data_len=%d",
-		  ssn, subflow->map_subflow_seq, subflow->map_data_len);
-}
-
 static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 			 int nonblock, int flags, int *addr_len)
 {
@@ -573,6 +476,7 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 
 	lock_sock(ssk);
 
+	arg.msg = msg;
 	desc.arg.data = &arg;
 	desc.error = 0;
 
@@ -581,120 +485,41 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 	len = min_t(size_t, len, INT_MAX);
 
 	while (copied < len) {
-		enum mapping_status status;
 		u32 map_remaining;
 		int bytes_read;
 		u64 ack_seq;
 		u64 old_ack;
-		u32 ssn;
 
 		smp_mb__before_atomic();
 		clear_bit(MPTCP_DATA_READY, &msk->flags);
 		smp_mb__after_atomic();
 
-		status = mptcp_get_mapping(ssk);
-
-		if (status == MAPPING_ADDED) {
-			/* Common case, but nothing to do here */
-		} else if (status == MAPPING_MISSING) {
-			struct sk_buff *skb = skb_peek(&ssk->sk_receive_queue);
-
-			if (!skb->len) {
-				/* the TCP stack deliver 0 len FIN pkt
-				 * to the receive queue, that is the only
-				 * 0len pkts ever expected here, and we can
-				 * admit no mapping only for 0 len pkts
-				 */
-				if (!(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN))
-					WARN_ONCE(1, "0len seq %d:%d flags %x",
-						  TCP_SKB_CB(skb)->seq,
-						  TCP_SKB_CB(skb)->end_seq,
-						  TCP_SKB_CB(skb)->tcp_flags);
-				sk_eat_skb(sk, skb);
-				continue;
-			}
-			if (!subflow->map_valid) {
-				WARN_ONCE(1, "corrupted stream: missing mapping");
-				copied = -EBADFD;
-				break;
-			}
-		} else if (status == MAPPING_EMPTY) {
+		if (!mptcp_subflow_data_available(ssk))
 			goto wait_for_data;
-		} else if (status == MAPPING_DATA_FIN) {
-			/* TODO: Handle according to RFC 6824 */
-			if (!copied) {
-				pr_err("Can't read after DATA_FIN");
-				copied = -ENOTCONN;
-			}
-
-			break;
-		}
 
-		ssn = tcp_sk(ssk)->copied_seq - subflow->ssn_offset;
 		old_ack = msk->ack_seq;
-
-		if (unlikely(before(ssn, subflow->map_subflow_seq))) {
-			/* Mapping covers data later in the subflow stream,
-			 * currently unsupported.
-			 */
-			warn_bad_map(subflow, ssn);
-			copied = -EBADFD;
-			break;
-		} else if (unlikely(!before(ssn, (subflow->map_subflow_seq +
-						  subflow->map_data_len)))) {
-			/* Mapping ends earlier in the subflow stream.
-			 * Invalid
-			 */
-			warn_bad_map(subflow, ssn);
-			copied = -EBADFD;
-			break;
-		}
-
 		ack_seq = mptcp_subflow_get_mapped_dsn(subflow);
 		map_remaining = subflow->map_data_len -
 				mptcp_subflow_get_map_offset(subflow);
 
-		if (before64(ack_seq, old_ack)) {
-			/* Mapping covers data already received, discard data
-			 * in the current mapping
-			 */
-			arg.msg = NULL;
-			desc.count = min_t(size_t, old_ack - ack_seq,
-					   map_remaining);
-			pr_debug("Dup data, map len=%d acked=%lld dropped=%zu",
-				 map_remaining, old_ack - ack_seq, desc.count);
-		} else {
-			arg.msg = msg;
-			desc.count = min_t(size_t, len - copied, map_remaining);
-		}
-
 		/* Read mapped data */
+		desc.count = min_t(size_t, len - copied, map_remaining);
 		bytes_read = tcp_read_sock(ssk, &desc, mptcp_read_actor);
 		if (bytes_read < 0)
 			break;
 
 		/* Refresh current MPTCP sequence number based on subflow seq */
 		ack_seq = mptcp_subflow_get_mapped_dsn(subflow);
-
 		if (before64(old_ack, ack_seq))
 			msk->ack_seq = ack_seq;
 
-		if (!before(tcp_sk(ssk)->copied_seq - subflow->ssn_offset,
-			    subflow->map_subflow_seq + subflow->map_data_len)) {
-			subflow->map_valid = 0;
-			pr_debug("Done with mapping: seq=%u data_len=%u",
-				 subflow->map_subflow_seq,
-				 subflow->map_data_len);
-		}
-
-		if (arg.msg)
-			copied += bytes_read;
+		copied += bytes_read;
 
 		/* The 'wait_for_data' code path can terminate the receive loop
 		 * in a number of scenarios: check if more data is pending
 		 * before giving up.
 		 */
-		if (!skb_queue_empty(&ssk->sk_receive_queue))
+		if (mptcp_subflow_data_available(ssk))
 			continue;
 
 wait_for_data:
net/mptcp/protocol.h +6 -4
index 98080d9f72fd..6e36207d9800 100644
@@ -204,8 +204,8 @@ struct mptcp_subflow_context {
 	u32     rel_write_seq;
 	u32	map_subflow_seq;
 	u32	ssn_offset;
-	u16	map_data_len;
-	u16	request_mptcp : 1,  /* send MP_CAPABLE */
+	u32	map_data_len;
+	u32	request_mptcp : 1,  /* send MP_CAPABLE */
 		request_join : 1,   /* send MP_JOIN */
 		request_bkup : 1,
 		request_version : 4,
@@ -214,7 +214,9 @@ struct mptcp_subflow_context {
 		fourth_ack : 1,     /* send initial DSS */
 		conn_finished : 1,
 		map_valid : 1,
-		backup : 1;
+		backup : 1,
+		data_avail : 1,
+		rx_eof : 1;
 	u32	remote_nonce;
 	u64	thmac;
 	u32	local_nonce;
@@ -260,7 +262,7 @@ mptcp_subflow_get_mapped_dsn(struct mptcp_subflow_context *subflow)
 }
 
 int mptcp_is_enabled(struct net *net);
-
+bool mptcp_subflow_data_available(struct sock *sk);
 void mptcp_subflow_init(void);
 int mptcp_subflow_connect(struct sock *sk, struct sockaddr_in *local,
 			  struct sockaddr_in *remote, u8 remote_id);
net/mptcp/subflow.c +267 -3

284 lines not on this page · raw message

Signature · 1 line
2.21.0
author
#6 lore raw

RFC PATCH 3/3 mptcp: recvmsg() can drain data from multiple subflows

In reply to Paolo Abeni

With the previous patch in place, the msk can detect which subflow has the current map with a simple walk, let's update the main loop to always select the 'current' subflow.

Additionally:
- changes the exit conditions to mirror closely tcp_recvmsg() this should fix unexpected behavior with timeout and signals.
- brings in MSG_WAITALL && lowlatency support. This is actually required to avoid deadlock in the selftests once we switch to the above end of main loop conditions

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/protocol.c | 174 +++++++++++++++++++++++++++----------------
 1 file changed, 111 insertions(+), 63 deletions(-)
net/mptcp/protocol.c +111 -63
index da2d749d69f4..21743afd935c 100644
@@ -92,6 +92,29 @@ static struct sock *mptcp_subflow_get(const struct mptcp_sock *msk)
 	return NULL;
 }
 
+static struct sock *mptcp_subflow_recv_lookup(const struct mptcp_sock *msk)
+{
+	struct sock *sk = (struct sock *)msk;
+	struct mptcp_subflow_context *subflow;
+	int receivers = 0;
+
+	sock_owned_by_me(sk);
+
+	mptcp_for_each_subflow(msk, subflow) {
+		if (subflow->data_avail)
+			return mptcp_subflow_tcp_socket(subflow)->sk;
+
+		receivers += !subflow->rx_eof;
+	}
+
+	/* hopefully temporary hack: propagate shutdown status from subflow
+	 * to msk, when all subflows agree on it
+	 */
+	if (!receivers && !(sk->sk_shutdown & RCV_SHUTDOWN))
+		sk->sk_shutdown |= RCV_SHUTDOWN;
+	return NULL;
+}
+
 static inline bool mptcp_skb_can_collapse_to(u64 write_seq,
 					     const struct sk_buff *skb,
 					     const struct mptcp_ext *mpext)
@@ -452,6 +475,7 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 	struct tcp_sock *tp;
 	struct sock *ssk;
 	int copied = 0;
+	int target;
 	long timeo;
 
 	lock_sock(sk);
@@ -465,17 +489,6 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 		return copied;
 	}
 
-	ssk = mptcp_subflow_get(msk);
-	if (!ssk) {
-		release_sock(sk);
-		return -ENOTCONN;
-	}
-
-	subflow = mptcp_subflow_ctx(ssk);
-	tp = tcp_sk(ssk);
-
-	lock_sock(ssk);
-
 	arg.msg = msg;
 	desc.arg.data = &arg;
 	desc.error = 0;
@@ -483,75 +496,102 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 	timeo = sock_rcvtimeo(sk, nonblock);
 
 	len = min_t(size_t, len, INT_MAX);
+	target = sock_rcvlowat(sk, flags & MSG_WAITALL, len);
 
 	while (copied < len) {
 		u32 map_remaining;
 		int bytes_read;
-		u64 ack_seq;
-		u64 old_ack;
 
 		smp_mb__before_atomic();
 		clear_bit(MPTCP_DATA_READY, &msk->flags);
 		smp_mb__after_atomic();
 
-		if (!mptcp_subflow_data_available(ssk))
+		ssk = mptcp_subflow_recv_lookup(msk);
+		pr_debug("msk=%p ssk=%p", msk, ssk);
+		if (!ssk)
 			goto wait_for_data;
 
-		old_ack = msk->ack_seq;
-		ack_seq = mptcp_subflow_get_mapped_dsn(subflow);
-		map_remaining = subflow->map_data_len -
-				mptcp_subflow_get_map_offset(subflow);
-
-		/* Read mapped data */
-		desc.count = min_t(size_t, len - copied, map_remaining);
-		bytes_read = tcp_read_sock(ssk, &desc, mptcp_read_actor);
-		if (bytes_read < 0)
-			break;
+		subflow = mptcp_subflow_ctx(ssk);
+		tp = tcp_sk(ssk);
 
-		/* Refresh current MPTCP sequence number based on subflow seq */
-		ack_seq = mptcp_subflow_get_mapped_dsn(subflow);
-		if (before64(old_ack, ack_seq))
-			msk->ack_seq = ack_seq;
-
-		copied += bytes_read;
-
-		/* The 'wait_for_data' code path can terminate the receive loop
-		 * in a number of scenarios: check if more data is pending
-		 * before giving up.
-		 */
-		if (mptcp_subflow_data_available(ssk))
-			continue;
+		lock_sock(ssk);
+		while (mptcp_subflow_data_available(ssk)) {
+			/* try to read as much data as available */
+			map_remaining = subflow->map_data_len -
+					mptcp_subflow_get_map_offset(subflow);
+			desc.count = min_t(size_t, len - copied, map_remaining);
+			pr_debug("reading %ld bytes, copied %d", desc.count,
+				 copied);
+			bytes_read = tcp_read_sock(ssk, &desc, mptcp_read_actor);
+			if (bytes_read < 0) {
+				if (!copied)
+					copied = bytes_read;
+				release_sock(ssk);
+				goto out;
+			}
+
+			pr_debug("msk ack_seq=%llx -> %llx", msk->ack_seq,
+				 msk->ack_seq + bytes_read);
+			msk->ack_seq += bytes_read;
+			copied += bytes_read;
+			if (copied >= len) {
+				release_sock(ssk);
+				goto out;
+			}
+			if (tp->urg_data && tp->urg_seq == tp->copied_seq) {
+				pr_err("Urgent data present, cannot proceed");
+				release_sock(ssk);
+				goto out;
+			}
+		}
+		release_sock(ssk);
+		continue;
 
 wait_for_data:
-		if (copied)
-			break;
-
-		if (tp->urg_data && tp->urg_seq == tp->copied_seq) {
-			pr_err("Urgent data present, cannot proceed");
+		/* only the master socket status is relevant here. The exit
+		 * conditions mirror closely tcp_recvmsg()
+		 */
+		if (copied >= target)
 			break;
-		}
 
-		if (ssk->sk_err || ssk->sk_state == TCP_CLOSE ||
-		    (ssk->sk_shutdown & RCV_SHUTDOWN) || !timeo ||
-		    signal_pending(current)) {
-			pr_debug("nonblock or error");
-			break;
+		if (copied) {
+			if (sk->sk_err ||
+			    sk->sk_state == TCP_CLOSE ||
+			    (sk->sk_shutdown & RCV_SHUTDOWN) ||
+			    !timeo ||
+			    signal_pending(current))
+				break;
+		} else {
+			if (sk->sk_err) {
+				copied = sock_error(sk);
+				break;
+			}
+
+			if (sk->sk_shutdown & RCV_SHUTDOWN)
+				break;
+
+			if (sk->sk_state == TCP_CLOSE) {
+				copied = -ENOTCONN;
+				break;
+			}
+
+			if (!timeo) {
+				copied = -EAGAIN;
+				break;
+			}
+
+			if (signal_pending(current)) {
+				copied = sock_intr_errno(timeo);
+				break;
+			}
 		}
 
-		/* Handle blocking and retry read if needed.
-		 *
-		 * Wait on MPTCP sock, the subflow will notify via data ready.
-		 */
-
-		pr_debug("block");
-		release_sock(ssk);
+		pr_debug("block timeout %ld", timeo);
 		mptcp_wait_data(sk, &timeo);
-		lock_sock(ssk);
 	}
 
-	release_sock(ssk);
+out:
 	release_sock(sk);
-
 	return copied;
 }
 
@@ -800,12 +840,7 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,
 		atomic64_set(&msk->snd_una, msk->write_seq);
 		ack_seq++;
 		msk->ack_seq = ack_seq;
-		subflow->map_seq = ack_seq;
-		subflow->map_subflow_seq = 1;
-		subflow->rel_write_seq = 1;
-		subflow->tcp_sock = new_sock;
 		newsk = new_mptcp_sock;
-		subflow->conn = new_mptcp_sock;
 		list_add(&subflow->node, &msk->conn_list);
 		bh_unlock_sock(new_mptcp_sock);
 
@@ -813,6 +848,19 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,
 		local_bh_enable();
 		inet_sk_state_store(newsk, TCP_ESTABLISHED);
 		release_sock(sk);
+
+		/* the subflow can already receive packet, avoid racing with
+		 * the receive path and process the pending ones
+		 */
+		lock_sock(new_sock->sk);
+		subflow->map_seq = ack_seq;
+		subflow->map_subflow_seq = 1;
+		subflow->rel_write_seq = 1;
+		subflow->tcp_sock = new_sock;
+		subflow->conn = new_mptcp_sock;
+		if (unlikely(!skb_queue_empty(&new_sock->sk->sk_receive_queue)))
+			mptcp_subflow_data_available(new_sock->sk);
+		release_sock(new_sock->sk);
 	} else {
 		newsk = new_sock->sk;
 		tcp_sk(newsk)->is_mptcp = 0;
Signature · 1 line
2.21.0
#3 lore raw

[MPTCP] Re: [RFC PATCH 2/3] mptcp: flush duplicate data at data_ready() time

In reply to Paolo Abeni
Quoted 14 lines · Paolo Abeni

Move subflow mapping related function to subflow.c - I think they belong there.

Refactor slightly get_mapping() to additionally validate the current mapping, moving some code from mptcp_recvmsg().

mptcp_data_ready() drops any skb which does not match exactly the current MPTCP seq number.

Old seqs are likely spurious retransmit we really want to ignore "future" seq may really happen in case of active backup switch-over, but we should converge to the current seq via 'correct' MPTCP-level retransmissions.

Quoted 6 lines
Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---

net/mptcp/protocol.c | 187 +-----------------------------
net/mptcp/protocol.h |  10 +-
net/mptcp/subflow.c  | 270 ++++++++++++++++++++++++++++++++++++++++++-
3 files changed, 279 insertions(+), 188 deletions(-)

(no comments for protocol.c)

net/mptcp/protocol.h struct mptcp_subflow_context { · line 207

Quoted 12 lines
net/mptcp/protocol.h +1 -2
index 98080d9f72fd..6e36207d9800 100644
@@ -204,8 +204,8 @@ struct mptcp_subflow_context {
 	u32     rel_write_seq;
 	u32	map_subflow_seq;
 	u32	ssn_offset;
-	u16	map_data_len;
-	u16	request_mptcp : 1,  /* send MP_CAPABLE */
+	u32	map_data_len;

Is this changed to u32 to fill the new hole in the data structure? The mapping length is 16 bits in RFC6824/6824bis.

Quoted 45 lines
net/mptcp/protocol.h +5 -2
+	u32	request_mptcp : 1,  /* send MP_CAPABLE */
 		request_join : 1,   /* send MP_JOIN */
 		request_bkup : 1,
 		request_version : 4,
@@ -214,7 +214,9 @@ struct mptcp_subflow_context {
 		fourth_ack : 1,     /* send initial DSS */
 		conn_finished : 1,
 		map_valid : 1,
-		backup : 1;
+		backup : 1,
+		data_avail : 1,
+		rx_eof : 1;
 	u32	remote_nonce;
 	u64	thmac;
 	u32	local_nonce;
@@ -260,7 +262,7 @@ mptcp_subflow_get_mapped_dsn(struct mptcp_subflow_context *subflow)
 }
 
 int mptcp_is_enabled(struct net *net);
-
+bool mptcp_subflow_data_available(struct sock *sk);
 void mptcp_subflow_init(void);
 int mptcp_subflow_connect(struct sock *sk, struct sockaddr_in *local,
 			  struct sockaddr_in *remote, u8 remote_id);
net/mptcp/subflow.c +13 -0
index ebe93525b0aa..e5ea2eba6537 100644
@@ -311,17 +311,281 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 
 static struct inet_connection_sock_af_ops subflow_specific;
 
+enum mapping_status {
+	MAPPING_OK,
+	MAPPING_INVALID,
+	MAPPING_EMPTY,
+	MAPPING_DATA_FIN
+};
+
+static u64 expand_seq(u64 old_seq, u16 old_data_len, u64 seq)
+{
+	if ((u32)seq == (u32)old_seq)
+		return old_seq;
+
+	/* Assume map covers data not mapped yet. */

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 336

Quoted 10 lines
net/mptcp/subflow.c +10 -0
+	return seq | ((old_seq + old_data_len + 1) & GENMASK_ULL(63, 32));
+}
+
+static void warn_bad_map(struct mptcp_subflow_context *subflow, u32 ssn)
+{
+	WARN_ONCE(1, "Bad mapping: ssn=%d map_seq=%d map_data_len=%d",
+		  ssn, subflow->map_subflow_seq, subflow->map_data_len);
+}
+
+static bool validate_mapping(struct sock *ssk, struct sk_buff *skb)

skb is unused.

Quoted 40 lines
net/mptcp/subflow.c +40 -0
+{
+	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
+	u32 ssn = tcp_sk(ssk)->copied_seq - subflow->ssn_offset;
+
+	if (unlikely(before(ssn, subflow->map_subflow_seq))) {
+		/* Mapping covers data later in the subflow stream,
+		 * currently unsupported.
+		 */
+		warn_bad_map(subflow, ssn);
+		return false;
+	}
+	if (unlikely(!before(ssn, (subflow->map_subflow_seq +
+					  subflow->map_data_len)))) {
+		/* Mapping ends earlier in the subflow stream. Invalid */
+		warn_bad_map(subflow, ssn);
+		return false;
+	}
+	return true;
+}
+
+static enum mapping_status get_mapping_status(struct sock *ssk)
+{
+	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
+	struct mptcp_ext *mpext;
+	struct sk_buff *skb;
+	u64 map_seq;
+
+	skb = skb_peek(&ssk->sk_receive_queue);
+	if (!skb)
+		return MAPPING_EMPTY;
+
+	mpext = mptcp_get_ext(skb);
+	if (!mpext || !mpext->use_map) {
+		if (!subflow->map_valid && !skb->len) {
+			/* the TCP stack deliver 0 len FIN pkt to the receive
+			* queue, that is the only 0len pkts ever expected here,
+			* and we can admit no mapping only for 0 len pkts
+			*/
+			if (!(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN))
+				WARN_ONCE(1, "0len seq %d:%d flags %x",

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 386

Quoted 10 lines
net/mptcp/subflow.c +10 -0
+					  TCP_SKB_CB(skb)->seq,
+					  TCP_SKB_CB(skb)->end_seq,
+					  TCP_SKB_CB(skb)->tcp_flags);
+			sk_eat_skb(ssk, skb);
+			return MAPPING_EMPTY;
+		}
+
+		if (!subflow->map_valid)
+			return MAPPING_INVALID;
+		goto validate_seq;

Is there a need to re-validate the existing mapping when subflow->map_valid is true and this function hasn't modified the mapping?

Quoted 25 lines
net/mptcp/subflow.c +25 -0
+	}
+
+	pr_debug("seq=%llu is64=%d ssn=%u data_len=%u", mpext->data_seq,
+		 mpext->dsn64, mpext->subflow_seq, mpext->data_len);
+
+	if (mpext->data_len == 0) {
+		pr_err("Infinite mapping not handled");
+		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_INFINITEMAPRX);
+		return MAPPING_INVALID;
+	} else if (mpext->subflow_seq == 0 &&
+		   mpext->data_fin == 1) {
+		if (WARN_ON_ONCE(mpext->data_len != 1))
+			return false;
+
+		pr_debug("DATA_FIN with no payload");
+		return MAPPING_OK;
+	}
+
+	if (!mpext->dsn64) {
+		map_seq = expand_seq(subflow->map_seq, subflow->map_data_len,
+				     mpext->data_seq);
+		pr_debug("expanded seq=%llu", subflow->map_seq);
+	} else {
+		map_seq = mpext->data_seq;
+	}

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 421

Quoted 10 lines
net/mptcp/subflow.c +10 -0
+
+	if (subflow->map_valid) {
+		/* check for in-sequence new map; since the current mapping
+		 * still valid, we can't replace it yet; instead increase the
+		 * current mapping size
+		 */
+		if (subflow->map_seq + subflow->map_data_len == mpext->data_seq) {
+			subflow->map_data_len += mpext->data_len;
+			goto validate_seq;
+		}

When checksums are added this gets more complicated, but I think it's ok for now.

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 423

Quoted 2 lines
net/mptcp/subflow.c +2 -0
+
+		/* Allow replacing only with a map extending the current one */

Regarding this comment: the code below is also intended to allow duplicate mappings (not only extension of the current one).

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 426

Quoted 3 lines
net/mptcp/subflow.c +3 -0
+		if (subflow->map_seq != mpext->data_seq ||
+		    subflow->map_subflow_seq != mpext->subflow_seq ||
+		    subflow->map_data_len < mpext->data_len) {

Should this instead invalidate the mapping when subflow->map_data_len > mpext->data_len ?

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 431

Quoted 5 lines
net/mptcp/subflow.c +5 -0
+			MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_DSSNOMATCH);
+			return MAPPING_INVALID;
+		}
+
+		subflow->map_data_len = mpext->data_len;

Does it make sense to add a skb_ext_del() here to avoid processing the DSS option again?

Quoted 67 lines
net/mptcp/subflow.c +67 -0
+		return MAPPING_OK;
+	}
+
+	subflow->map_seq = map_seq;
+	subflow->map_subflow_seq = mpext->subflow_seq;
+	subflow->map_data_len = mpext->data_len;
+	subflow->map_valid = 1;
+	pr_debug("new map seq=%llu subflow_seq=%u data_len=%u",
+		 subflow->map_seq, subflow->map_subflow_seq,
+		 subflow->map_data_len);
+
+validate_seq:
+	if (!validate_mapping(ssk, skb))
+		return MAPPING_INVALID;
+
+	skb_ext_del(skb, SKB_EXT_MPTCP);
+	return MAPPING_OK;
+}
+
+static bool subflow_check_data_avail(struct sock *ssk)
+{
+	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
+	enum mapping_status status;
+	struct mptcp_sock *msk;
+	struct sk_buff *skb;
+
+	pr_debug("msk=%p ssk=%p data_avail=%d skb=%p", subflow->conn, ssk,
+		 subflow->data_avail, skb_peek(&ssk->sk_receive_queue));
+	if (subflow->data_avail)
+		return true;
+
+	if (!subflow->conn)
+		return false;
+
+	msk = mptcp_sk(subflow->conn);
+	for (;;) {
+		u32 map_remaining;
+		size_t delta;
+		u64 ack_seq;
+		u64 old_ack;
+
+		status = get_mapping_status(ssk);
+		pr_debug("msk=%p ssk=%p status=%d", msk, ssk, status);
+		if (status == MAPPING_INVALID) {
+			ssk->sk_err = EBADMSG;
+			goto fatal;
+		}
+
+		if (status != MAPPING_OK)
+			return false;
+
+		skb = skb_peek(&ssk->sk_receive_queue);
+		if (WARN_ON_ONCE(!skb))
+			return false;
+
+		old_ack = READ_ONCE(msk->ack_seq);
+		ack_seq = mptcp_subflow_get_mapped_dsn(subflow);
+		pr_debug("msk ack_seq=%llx subflow ack_seq=%llx", old_ack,
+			 ack_seq);
+		if (ack_seq == old_ack)
+			break;
+
+		/* only accept in-sequence mapping. Old values are spurious
+		 * retransmission; we can hit "future" values on active backup
+		 * subflow switch, we relay on retransmissions to get
+		 * in-sequence data.
+		 * Cuncurrent subflows support will require subflow data

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 508

Quoted 10 lines
net/mptcp/subflow.c +10 -0
+		 * reordering
+		 */
+		map_remaining = subflow->map_data_len -
+				mptcp_subflow_get_map_offset(subflow);
+		if (before64(ack_seq, old_ack))
+			delta = min_t(size_t, old_ack - ack_seq, map_remaining);
+		else
+			delta = min_t(size_t, ack_seq - old_ack, map_remaining);
+
+		/* discard mapping data */

s/mapping/mapped/

Overall, the refactoring is an improvement. Thanks!

Mat

Quoted 90 lines
net/mptcp/subflow.c +72 -3
+		pr_debug("discarding %ld bytes, current map len=%d", delta,
+			 map_remaining);
+		if (delta) {
+			struct mptcp_read_arg arg = {
+				.msg = NULL,
+			};
+			read_descriptor_t desc = {
+				.count = delta,
+				.arg.data = &arg,
+			};
+			int ret;
+
+			ret = tcp_read_sock(ssk, &desc, mptcp_read_actor);
+			if (ret < 0) {
+				ssk->sk_err = -ret;
+				goto fatal;
+			}
+			if (ret < delta)
+				return false;
+			if (delta == map_remaining)
+				subflow->map_valid = 0;
+		}
+	}
+	return true;
+
+fatal:
+	/* fatal protocol error, close the socket */
+	/* This barrier is coupled with smp_rmb() in tcp_poll() */
+	smp_wmb();
+	ssk->sk_error_report(ssk);
+	tcp_set_state(ssk, TCP_CLOSE);
+	tcp_send_active_reset(ssk, GFP_ATOMIC);
+	return false;
+}
+
+bool mptcp_subflow_data_available(struct sock *sk)
+{
+	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
+	struct sk_buff *skb;
+
+	/* check if current mapping is still valid */
+	if (subflow->map_valid &&
+	    mptcp_subflow_get_map_offset(subflow) >= subflow->map_data_len) {
+		subflow->map_valid = 0;
+		subflow->data_avail = 0;
+
+		pr_debug("Done with mapping: seq=%u data_len=%u",
+			 subflow->map_subflow_seq,
+			 subflow->map_data_len);
+	}
+
+	if (!subflow_check_data_avail(sk)) {
+		/* set EoF only there is no data available - we already spooled
+		 * all the pending skbs
+		 */
+		if (sk->sk_shutdown & RCV_SHUTDOWN || sk->sk_state == TCP_CLOSE)
+			subflow->rx_eof = 1;
+		return false;
+	}
+
+	skb = skb_peek(&sk->sk_receive_queue);
+	if (!skb)
+		return false;
+
+	subflow->data_avail = tcp_sk(sk)->copied_seq < TCP_SKB_CB(skb)->end_seq;
+	return subflow->data_avail;
+}
+
 static void subflow_data_ready(struct sock *sk)
 {
 	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
 	struct sock *parent = subflow->conn;
 
-	pr_debug("sk=%p", sk);
 	subflow->tcp_sk_data_ready(sk);
 
-	if (parent) {
-		pr_debug("parent=%p", parent);
+	if (!parent || !(subflow->mp_capable || subflow->mp_join))
+		return;
 
+	/* always propagate the EoF */
+	if (mptcp_subflow_data_available(sk) || subflow->rx_eof) {
 		smp_mb__before_atomic();
 		set_bit(MPTCP_DATA_READY, &mptcp_sk(parent)->flags);
 		smp_mb__after_atomic();
Signature · 1 line
2.21.0

--
Mat Martineau
Intel

#7 lore raw
In reply to Paolo Abeni
Quoted 11 lines · Paolo Abeni

This start constructing the instrucutre to support pulling data from multiple subflow. At the current stage uses a very simply approach, dropping any out-of-order data and reading from any subflow with valid data.

The above should work quite nicely in active backup scenario and possibly even with multiple subflows sending data concurrently - with a low bwidth and a huge number of retransmissions.

This is a not_so_early draft, shared as it may pave some ground and/or partially overlap with DATA_FIN support effort

It definitely overlaps with the DATA_FIN changes, but please keep moving ahead. I will rebase my DATA_FIN work on your refactored code.

Mat

Quoted 9 lines

Note that this rewrite completely the recvmsg() main loop to mirror more closely the TCP recvmsg() main loop, likely fixing some bug in respect to signal and error condition handling.

Paolo Abeni (3):
mptcp: move some helper into the header file
mptcp: flush duplicate data at data_ready() time
mptcp: recvmsg() can drain data from multiple subflows

Quoted 9 lines
net/mptcp/protocol.c | 360 +++++++++++++------------------------------
net/mptcp/protocol.h |  31 +++-
net/mptcp/subflow.c  | 270 +++++++++++++++++++++++++++++++-
3 files changed, 403 insertions(+), 258 deletions(-)
Signature · 1 line
2.21.0

--
Mat Martineau
Intel

author
#4 lore raw

[MPTCP] Re: [RFC PATCH 2/3] mptcp: flush duplicate data at data_ready() time

In reply to Mat Martineau

Hi,

Thank you for the review.

On Mon, 2019-10-14 at 16:28 -0700, Mat Martineau wrote:
[...]

Quoted 6 lines
net/mptcp/protocol.h +0 -0
Quoted 5 lines

index 98080d9f72fd..6e36207d9800 100644

net/mptcp/protocol.h +0 -0
@@ -204,8 +204,8 @@ struct mptcp_subflow_context {
 	u32     rel_write_seq;
Quoted 8 lines

net/mptcp/protocol.h struct mptcp_subflow_context { · line 207

Quoted 5 lines
net/mptcp/protocol.h +1 -2
 	u32	map_subflow_seq;
 	u32	ssn_offset;
-	u16	map_data_len;
-	u16	request_mptcp : 1,  /* send MP_CAPABLE */
+	u32	map_data_len;

Is this changed to u32 to fill the new hole in the data structure? The mapping length is 16 bits in RFC6824/6824bis.

This is needed because this patch also introduces trivial mapping coalescing, a few hunks below. It also will allow more extensive skb colaescing some day ;)

Quoted 16 lines
Quoted 16 lines
net/mptcp/protocol.h +4 -1
+	u32	request_mptcp : 1,  /* send MP_CAPABLE */
 		request_join : 1,   /* send MP_JOIN */
 		request_bkup : 1,
 		request_version : 4,
@@ -214,7 +214,9 @@ struct mptcp_subflow_context {
 		fourth_ack : 1,     /* send initial DSS */
 		conn_finished : 1,
 		map_valid : 1,
-		backup : 1;
+		backup : 1,
+		data_avail : 1,
+		rx_eof : 1;
 	u32	remote_nonce;
 	u64	thmac;
 	u32	local_nonce;

[...]

Quoted 25 lines
Quoted 25 lines
net/mptcp/protocol.h +25 -0
+static enum mapping_status get_mapping_status(struct sock *ssk)
+{
+	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
+	struct mptcp_ext *mpext;
+	struct sk_buff *skb;
+	u64 map_seq;
+
+	skb = skb_peek(&ssk->sk_receive_queue);
+	if (!skb)
+		return MAPPING_EMPTY;
+
+	mpext = mptcp_get_ext(skb);
+	if (!mpext || !mpext->use_map) {
+		if (!subflow->map_valid && !skb->len) {
+			/* the TCP stack deliver 0 len FIN pkt to the receive
+			* queue, that is the only 0len pkts ever expected here,
+			* and we can admit no mapping only for 0 len pkts
+			*/
+			if (!(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN))
+				WARN_ONCE(1, "0len seq %d:%d flags %x",
+					  TCP_SKB_CB(skb)->seq,
+					  TCP_SKB_CB(skb)->end_seq,
+					  TCP_SKB_CB(skb)->tcp_flags);
+			sk_eat_skb(ssk, skb);
+			return MAPPING_EMPTY;
Quoted 8 lines

net/mptcp/protocol.h

Quoted 5 lines
net/mptcp/protocol.h +5 -0
+		}
+
+		if (!subflow->map_valid)
+			return MAPPING_INVALID;
+		goto validate_seq;

Is there a need to re-validate the existing mapping when subflow->map_valid is true and this function hasn't modified the mapping?

I'm not actually 110% this is strictly necessary, but I introduced that as an additional safeguards ad validate_mapping() depends on the current ssn, and such sequence can be changed since last validation, as it's cheap.

Quoted 30 lines
Quoted 30 lines
net/mptcp/protocol.h +30 -0
+	}
+
+	pr_debug("seq=%llu is64=%d ssn=%u data_len=%u", mpext->data_seq,
+		 mpext->dsn64, mpext->subflow_seq, mpext->data_len);
+
+	if (mpext->data_len == 0) {
+		pr_err("Infinite mapping not handled");
+		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_INFINITEMAPRX);
+		return MAPPING_INVALID;
+	} else if (mpext->subflow_seq == 0 &&
+		   mpext->data_fin == 1) {
+		if (WARN_ON_ONCE(mpext->data_len != 1))
+			return false;
+
+		pr_debug("DATA_FIN with no payload");
+		return MAPPING_OK;
+	}
+
+	if (!mpext->dsn64) {
+		map_seq = expand_seq(subflow->map_seq, subflow->map_data_len,
+				     mpext->data_seq);
+		pr_debug("expanded seq=%llu", subflow->map_seq);
+	} else {
+		map_seq = mpext->data_seq;
+	}
+
+	if (subflow->map_valid) {
+		/* check for in-sequence new map; since the current mapping
+		 * still valid, we can't replace it yet; instead increase the
+		 * current mapping size
Quoted 8 lines

net/mptcp/protocol.h

Quoted 5 lines
net/mptcp/protocol.h +5 -0
+		 */
+		if (subflow->map_seq + subflow->map_data_len == mpext->data_seq) {
+			subflow->map_data_len += mpext->data_len;
+			goto validate_seq;
+		}

When checksums are added this gets more complicated, but I think it's ok for now.

yep! gladly we don't have that now ;)

Quoted 6 lines

net/mptcp/protocol.h

Quoted 2 lines
net/mptcp/protocol.h +2 -0
+
+		/* Allow replacing only with a map extending the current one */

Regarding this comment: the code below is also intended to allow duplicate mappings (not only extension of the current one).

Quoted 6 lines

net/mptcp/protocol.h

Quoted 3 lines
net/mptcp/protocol.h +3 -0
+		if (subflow->map_seq != mpext->data_seq ||
+		    subflow->map_subflow_seq != mpext->subflow_seq ||
+		    subflow->map_data_len < mpext->data_len) {

Should this instead invalidate the mapping when subflow->map_data_len > mpext->data_len ?

uhmm... this code followed some discussion on the ML and a few mtgs ago. My [mis]understanding is that we can[/want] be quite strict about map replacing: allowing it only if the old and the new maps are identical.

With the proposed alternative check we will allow also several kind of partial overlaps. I think could still be safe, but I would opt for a more conservative approach, if possible.

The implemented condition a bit weeker of my understanding of the preferred option, allowing the replace even if the new map completely overlap with the old one and additionally extends it. I did that because - due to some bug in the previous private version of these patches - I stumbled upon the above condition.

I can adapt to the stricter rule, if agreed.

Quoted 8 lines

net/mptcp/protocol.h

Quoted 5 lines
net/mptcp/protocol.h +5 -0
+			MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_DSSNOMATCH);
+			return MAPPING_INVALID;
+		}
+
+		subflow->map_data_len = mpext->data_len;

Does it make sense to add a skb_ext_del() here to avoid processing the DSS option again?

Indeed! will do in the next iteration.

Note: I think we get the greater performance benefit here from hitting more consistently the slab fast path for skb ext life cycle.

[...]

Quoted 9 lines
net/mptcp/protocol.h +1 -0
+		/* only accept in-sequence mapping. Old values are spurious
Quoted 8 lines
net/mptcp/protocol.h +8 -0
+		 * retransmission; we can hit "future" values on active backup
+		 * subflow switch, we relay on retransmissions to get
+		 * in-sequence data.
+		 * Cuncurrent subflows support will require subflow data
+		 * reordering
+		 */
+		map_remaining = subflow->map_data_len -
+				mptcp_subflow_get_map_offset(subflow);
Quoted 8 lines

net/mptcp/protocol.h

Quoted 6 lines
net/mptcp/protocol.h +6 -0
+		if (before64(ack_seq, old_ack))
+			delta = min_t(size_t, old_ack - ack_seq, map_remaining);
+		else
+			delta = min_t(size_t, ack_seq - old_ack, map_remaining);
+
+		/* discard mapping data */

s/mapping/mapped/

Yep, thanks. will cover in the next iteration!

cheers,

Paolo

author
#5 lore raw

[MPTCP] Re: [RFC PATCH 2/3] mptcp: flush duplicate data at data_ready() time

Quoted 8 lines · Paolo Abeni
Quoted 6 lines · Mat Martineau

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 426

Quoted 3 lines
net/mptcp/subflow.c +3 -0
+		if (subflow->map_seq != mpext->data_seq ||
+		    subflow->map_subflow_seq != mpext->subflow_seq ||
+		    subflow->map_data_len < mpext->data_len) {

Should this instead invalidate the mapping when subflow->map_data_len > mpext->data_len ?

Quoted 4 lines

uhmm... this code followed some discussion on the ML and a few mtgs ago. My [mis]understanding is that we can[/want] be quite strict about map replacing: allowing it only if the old and the new maps are identical.

I'm sorry, I misread your comment. Please ignore my previous answer.

Yep, I agree invalidating the mapping even with subflow->map_data_len > mpext->data_len, I'll do in the next iteration.

Thanks!

/P