Skip to contentSkip to conversation

mptcp: partial recvmsg() refactor

PATCH Superseded Paolo Abeni 2 participants 8 messages Sep 13, 2019 – Sep 18, 2019

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Paolo Abeni

author
#0 lore raw

cover mptcp: partial recvmsg() refactor

This 2 patches start the recvmsg() refactor discussed in last mtg. The explicit 'duplicate data' handling in 1/2 fix some stream corruption I observe with re-injection, and the 2nd one fix another stream corruption at the end of the stream, still triggered by re-injection.

With the above 2 patches I still stream corruptions with re-injection, are almost gone - even if I can still hit them with in at least a very rare corner case.

Paolo Abeni (2):
mptcp: refactor recvmsg()
mptcp: recvmsg() should really try to deliver 'len' bytes to
user-space

 net/mptcp/protocol.c | 119 +++++++++++++++++++------------------------
 1 file changed, 53 insertions(+), 66 deletions(-)
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH 1/2 mptcp: refactor recvmsg()

The current implementation tries to handle exceptional sitations (no mapping, bad subflow sequence number) without baling out, but that makes debugging existing bugs very difficult.

Also duplicate MPTCP data due to retransmissions are hard to track, because we discard the duplicate mapping and skip the following pkts lacking the map without any fourther control.

This changeset refactor the main loop to return an error code to the user space for unhandled exceptional condition. It also track explicitly the number of duplicate MPTCP data bytes to be discarded: duplicate mapping are still considered valid, and we just discard the bytes belonging to such mapping.

This makes debugging easier, and also clean-up a bit the existing code

Signed-off-by: Paolo Abeni <pabeni(a)redhat.com>
--
I'm ok to squash this into commit 786f07cf97f1 ("mptcp: Implement MPTCP receive path")

---
 net/mptcp/protocol.c | 112 ++++++++++++++++++-------------------------
 1 file changed, 46 insertions(+), 66 deletions(-)
net/mptcp/protocol.c +46 -66
index fba7e00df4ff..4fa4e07df17c 100644
@@ -452,17 +452,6 @@ static int mptcp_read_actor(read_descriptor_t *desc, struct sk_buff *skb,
 	return copy_len;
 }
 
-static int mptcp_flush_actor(read_descriptor_t *desc, struct sk_buff *skb,
-			     unsigned int offset, size_t len)
-{
-	pr_debug("Flushing one skb with %zu of %zu bytes remaining",
-		 len, len + offset);
-
-	desc->count = 0;
-
-	return len;
-}
-
 enum mapping_status {
 	MAPPING_ADDED,
 	MAPPING_MISSING,
@@ -611,7 +600,6 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 
 	while (copied < len) {
 		enum mapping_status status;
-		size_t discard_len = 0;
 		u32 map_remaining;
 		int bytes_read;
 		u64 ack_seq;
@@ -627,20 +615,27 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 		if (status == MAPPING_ADDED) {
 			/* Common case, but nothing to do here */
 		} else if (status == MAPPING_MISSING) {
-			if (!subflow->map_valid) {
-				pr_debug("Mapping missing, trying next skb");
-
-				arg.msg = NULL;
-				desc.count = SIZE_MAX;
-
-				bytes_read = tcp_read_sock(ssk, &desc,
-							   mptcp_flush_actor);
-
-				if (bytes_read < 0)
-					break;
+			struct sk_buff *skb = skb_peek(&ssk->sk_receive_queue);
 
+			if (!skb->len) {
+				/* the TCP stack deliver 0 len FIN pkt
+				 * to the receive queue, that is the only
+				 * 0len pkts ever expected here, and we can
+				 * admit no mapping only for 0 len pkts
+				 */
+				if (!(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN))
+					WARN_ONCE(1, "0len seq %d:%d flags %x",
+						  TCP_SKB_CB(skb)->seq,
+						  TCP_SKB_CB(skb)->end_seq,
+						  TCP_SKB_CB(skb)->tcp_flags);
+				sk_eat_skb(sk, skb);
 				continue;
 			}
+			if (!subflow->map_valid) {
+				WARN_ONCE(1, "corrupted stream: missing mapping");
+				copied = -EBADFD;
+				break;
+			}
 		} else if (status == MAPPING_EMPTY) {
 			goto wait_for_data;
 		} else if (status == MAPPING_DATA_FIN) {
@@ -658,60 +653,44 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 
 		if (unlikely(before(ssn, subflow->map_subflow_seq))) {
 			/* Mapping covers data later in the subflow stream,
-			 * discard unmapped data.
+			 * currently unsupported.
 			 */
-			pr_debug("Mapping covers data later in stream");
-			discard_len = subflow->map_subflow_seq - ssn;
+			WARN_ONCE(1, "Mapping covers data later in stream");
+			copied = -EBADFD;
+			break;
 		} else if (unlikely(!before(ssn, (subflow->map_subflow_seq +
 						  subflow->map_data_len)))) {
 			/* Mapping ends earlier in the subflow stream.
-			 * Invalidate the mapping and try again.
+			 * Invalid
 			 */
-			subflow->map_valid = 0;
-			pr_debug("Invalid mapping ssn=%d map_seq=%d map_data_len=%d",
-				 ssn, subflow->map_subflow_seq,
-				 subflow->map_data_len);
-			continue;
-		} else {
-			ack_seq = get_mapped_dsn(subflow);
-
-			if (before64(ack_seq, old_ack)) {
-				/* Mapping covers data already received,
-				 * discard data in the current mapping
-				 * and invalidate the map
-				 */
-				u64 map_end_dsn = subflow->map_seq +
-					subflow->map_data_len;
-				discard_len = min(map_end_dsn - ack_seq,
-						  old_ack - ack_seq);
-				subflow->map_valid = 0;
-				pr_debug("Duplicate MPTCP data found");
-			}
+			WARN_ONCE(1, "Mapping covers old data ssn=%d "
+				  "map_seq=%d map_data_len=%d", ssn,
+				  subflow->map_subflow_seq,
+				  subflow->map_data_len);
+			copied = -EBADFD;
+			break;
 		}
 
-		if (discard_len) {
-			/* Discard data for the current mapping.
-			 */
-			pr_debug("Discard %zu bytes", discard_len);
+		ack_seq = get_mapped_dsn(subflow);
+		map_remaining = subflow->map_data_len - get_map_offset(subflow);
 
+		if (before64(ack_seq, old_ack)) {
+			/* Mapping covers data already received, discard data
+			 * in the current mapping
+			 */
 			arg.msg = NULL;
-			desc.count = discard_len;
-
-			bytes_read = tcp_read_sock(ssk, &desc,
-						   mptcp_read_actor);
-
-			if (bytes_read < 0)
-				break;
-			else if (bytes_read == discard_len)
-				continue;
-			else
-				goto wait_for_data;
+			desc.count = min_t(size_t, old_ack - ack_seq,
+					   map_remaining);
+			pr_debug("Duplicate MPTCP data found, map "
+				 "remaining %d already acked data %lld "
+				 "discarding %ld bytes", map_remaining,
+				 old_ack - ack_seq, desc.count);
+		} else {
+			arg.msg = msg;
+			desc.count = min_t(size_t, len - copied, map_remaining);
 		}
 
 		/* Read mapped data */
-		map_remaining = subflow->map_data_len - get_map_offset(subflow);
-		desc.count = min_t(size_t, len - copied, map_remaining);
-		arg.msg = msg;
 		bytes_read = tcp_read_sock(ssk, &desc, mptcp_read_actor);
 		if (bytes_read < 0)
 			break;
@@ -730,7 +709,8 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 				 subflow->map_data_len);
 		}
 
-		copied += bytes_read;
+		if (arg.msg)
+			copied += bytes_read;
 
 wait_for_data:
 		if (copied)
Signature · 1 line
2.21.0
author
#6 lore raw

PATCH 2/2 mptcp: recvmsg() should really try to deliver 'len' bytes to user-space

In reply to Paolo Abeni

Currently we break the recvmsg() main loop as soon as we have copied any amount of data or the ssk is closed.

With reinjection we can receive duplicate data and reach the 'wait_for_data' label with 'copied == 0'.
The subflow sk could be already in closed status, if the local socket has already received all the data from the peer (and the data is sitting into the receive queue).

When we meet the above scenario, currently recvmsg() terminates with return code '0' and the user space interpret that as EoF, even if there is potentially a lot of data pending.

This change addresses the issue checking for pending data before going through the 'wait_for_data' code path.

Signed-off-by: Paolo Abeni <pabeni(a)redhat.com>
--
I'm ok with squashing to commit 786f07cf97f1 ("mptcp: Implement MPTCP receive path")

---
 net/mptcp/protocol.c | 7 +++++++
 1 file changed, 7 insertions(+)
net/mptcp/protocol.c +7 -0
index 4fa4e07df17c..402a43d5dac5 100644
@@ -712,6 +712,13 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 		if (arg.msg)
 			copied += bytes_read;
 
+		/* The 'wait_for_data' code path can terminte the receive loop
+		 * in a number of scenarios: check if more data is pending
+		 * before giving up.
+		 */
+		if (!skb_queue_empty(&ssk->sk_receive_queue))
+			continue;
+
 wait_for_data:
 		if (copied)
 			break;
Signature · 1 line
2.21.0
#2 lore raw

Re: [MPTCP] [PATCH 1/2] mptcp: refactor recvmsg()

In reply to Paolo Abeni
Quoted 8 lines · Paolo Abeni

The current implementation tries to handle exceptional sitations (no mapping, bad subflow sequence number) without baling out, but that makes debugging existing bugs very difficult.

Also duplicate MPTCP data due to retransmissions are hard to track, because we discard the duplicate mapping and skip the following pkts lacking the map without any fourther control.

Quoted 8 lines

This changeset refactor the main loop to return an error code to the user space for unhandled exceptional condition. It also track explicitly the number of duplicate MPTCP data bytes to be discarded: duplicate mapping are still considered valid, and we just discard the bytes belonging to such mapping.

This makes debugging easier, and also clean-up a bit the existing code

Thanks for doing this Paolo. I agree that it's better to handle the mappings covering duplicate MPTCP data like this, we don't want part of the duplicate data to look like unmapped data.

I'll run some tests with the multipath-tcp.org kernel tomorrow to finish up my review.

Mat

Quoted 171 lines

Signed-off-by: Paolo Abeni <pabeni(a)redhat.com>
--
I'm ok to squash this into commit 786f07cf97f1 ("mptcp: Implement MPTCP receive path")
---
net/mptcp/protocol.c | 112 ++++++++++++++++++-------------------------

1 file changed, 46 insertions(+), 66 deletions(-)
net/mptcp/protocol.c +46 -66
index fba7e00df4ff..4fa4e07df17c 100644
@@ -452,17 +452,6 @@ static int mptcp_read_actor(read_descriptor_t *desc, struct sk_buff *skb,
 	return copy_len;
 }
 
-static int mptcp_flush_actor(read_descriptor_t *desc, struct sk_buff *skb,
-			     unsigned int offset, size_t len)
-{
-	pr_debug("Flushing one skb with %zu of %zu bytes remaining",
-		 len, len + offset);
-
-	desc->count = 0;
-
-	return len;
-}
-
 enum mapping_status {
 	MAPPING_ADDED,
 	MAPPING_MISSING,
@@ -611,7 +600,6 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 
 	while (copied < len) {
 		enum mapping_status status;
-		size_t discard_len = 0;
 		u32 map_remaining;
 		int bytes_read;
 		u64 ack_seq;
@@ -627,20 +615,27 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 		if (status == MAPPING_ADDED) {
 			/* Common case, but nothing to do here */
 		} else if (status == MAPPING_MISSING) {
-			if (!subflow->map_valid) {
-				pr_debug("Mapping missing, trying next skb");
-
-				arg.msg = NULL;
-				desc.count = SIZE_MAX;
-
-				bytes_read = tcp_read_sock(ssk, &desc,
-							   mptcp_flush_actor);
-
-				if (bytes_read < 0)
-					break;
+			struct sk_buff *skb = skb_peek(&ssk->sk_receive_queue);
 
+			if (!skb->len) {
+				/* the TCP stack deliver 0 len FIN pkt
+				 * to the receive queue, that is the only
+				 * 0len pkts ever expected here, and we can
+				 * admit no mapping only for 0 len pkts
+				 */
+				if (!(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN))
+					WARN_ONCE(1, "0len seq %d:%d flags %x",
+						  TCP_SKB_CB(skb)->seq,
+						  TCP_SKB_CB(skb)->end_seq,
+						  TCP_SKB_CB(skb)->tcp_flags);
+				sk_eat_skb(sk, skb);
 				continue;
 			}
+			if (!subflow->map_valid) {
+				WARN_ONCE(1, "corrupted stream: missing mapping");
+				copied = -EBADFD;
+				break;
+			}
 		} else if (status == MAPPING_EMPTY) {
 			goto wait_for_data;
 		} else if (status == MAPPING_DATA_FIN) {
@@ -658,60 +653,44 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 
 		if (unlikely(before(ssn, subflow->map_subflow_seq))) {
 			/* Mapping covers data later in the subflow stream,
-			 * discard unmapped data.
+			 * currently unsupported.
 			 */
-			pr_debug("Mapping covers data later in stream");
-			discard_len = subflow->map_subflow_seq - ssn;
+			WARN_ONCE(1, "Mapping covers data later in stream");
+			copied = -EBADFD;
+			break;
 		} else if (unlikely(!before(ssn, (subflow->map_subflow_seq +
 						  subflow->map_data_len)))) {
 			/* Mapping ends earlier in the subflow stream.
-			 * Invalidate the mapping and try again.
+			 * Invalid
 			 */
-			subflow->map_valid = 0;
-			pr_debug("Invalid mapping ssn=%d map_seq=%d map_data_len=%d",
-				 ssn, subflow->map_subflow_seq,
-				 subflow->map_data_len);
-			continue;
-		} else {
-			ack_seq = get_mapped_dsn(subflow);
-
-			if (before64(ack_seq, old_ack)) {
-				/* Mapping covers data already received,
-				 * discard data in the current mapping
-				 * and invalidate the map
-				 */
-				u64 map_end_dsn = subflow->map_seq +
-					subflow->map_data_len;
-				discard_len = min(map_end_dsn - ack_seq,
-						  old_ack - ack_seq);
-				subflow->map_valid = 0;
-				pr_debug("Duplicate MPTCP data found");
-			}
+			WARN_ONCE(1, "Mapping covers old data ssn=%d "
+				  "map_seq=%d map_data_len=%d", ssn,
+				  subflow->map_subflow_seq,
+				  subflow->map_data_len);
+			copied = -EBADFD;
+			break;
 		}
 
-		if (discard_len) {
-			/* Discard data for the current mapping.
-			 */
-			pr_debug("Discard %zu bytes", discard_len);
+		ack_seq = get_mapped_dsn(subflow);
+		map_remaining = subflow->map_data_len - get_map_offset(subflow);
 
+		if (before64(ack_seq, old_ack)) {
+			/* Mapping covers data already received, discard data
+			 * in the current mapping
+			 */
 			arg.msg = NULL;
-			desc.count = discard_len;
-
-			bytes_read = tcp_read_sock(ssk, &desc,
-						   mptcp_read_actor);
-
-			if (bytes_read < 0)
-				break;
-			else if (bytes_read == discard_len)
-				continue;
-			else
-				goto wait_for_data;
+			desc.count = min_t(size_t, old_ack - ack_seq,
+					   map_remaining);
+			pr_debug("Duplicate MPTCP data found, map "
+				 "remaining %d already acked data %lld "
+				 "discarding %ld bytes", map_remaining,
+				 old_ack - ack_seq, desc.count);
+		} else {
+			arg.msg = msg;
+			desc.count = min_t(size_t, len - copied, map_remaining);
 		}
 
 		/* Read mapped data */
-		map_remaining = subflow->map_data_len - get_map_offset(subflow);
-		desc.count = min_t(size_t, len - copied, map_remaining);
-		arg.msg = msg;
 		bytes_read = tcp_read_sock(ssk, &desc, mptcp_read_actor);
 		if (bytes_read < 0)
 			break;
@@ -730,7 +709,8 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 				 subflow->map_data_len);
 		}
 
-		copied += bytes_read;
+		if (arg.msg)
+			copied += bytes_read;
 
Quoted 10 lines

wait_for_data:
if (copied)

Signature · 6 lines
2.21.0

_______________________________________________
mptcp mailing list
mptcp(a)lists.01.org
https://lists.01.org/mailman/listinfo/mptcp

--
Mat Martineau
Intel

#3 lore raw

Re: [MPTCP] [PATCH 1/2] mptcp: refactor recvmsg()

In reply to Paolo Abeni
Quoted 8 lines · Paolo Abeni

The current implementation tries to handle exceptional sitations (no mapping, bad subflow sequence number) without baling out, but that makes debugging existing bugs very difficult.

Also duplicate MPTCP data due to retransmissions are hard to track, because we discard the duplicate mapping and skip the following pkts lacking the map without any fourther control.

Quoted 8 lines

This changeset refactor the main loop to return an error code to the user space for unhandled exceptional condition. It also track explicitly the number of duplicate MPTCP data bytes to be discarded: duplicate mapping are still considered valid, and we just discard the bytes belonging to such mapping.

This makes debugging easier, and also clean-up a bit the existing code

Looks fine in terms of interoperability with v0.95. I am noticing a lot more reinjection by v0.95 than there used to be, but that happens with or without these patches.

Two minor formatting issues below.

Quoted 106 lines

Signed-off-by: Paolo Abeni <pabeni(a)redhat.com>
--
I'm ok to squash this into commit 786f07cf97f1 ("mptcp: Implement MPTCP receive path")
---
net/mptcp/protocol.c | 112 ++++++++++++++++++-------------------------

1 file changed, 46 insertions(+), 66 deletions(-)
net/mptcp/protocol.c +23 -39
index fba7e00df4ff..4fa4e07df17c 100644
@@ -452,17 +452,6 @@ static int mptcp_read_actor(read_descriptor_t *desc, struct sk_buff *skb,
 	return copy_len;
 }
 
-static int mptcp_flush_actor(read_descriptor_t *desc, struct sk_buff *skb,
-			     unsigned int offset, size_t len)
-{
-	pr_debug("Flushing one skb with %zu of %zu bytes remaining",
-		 len, len + offset);
-
-	desc->count = 0;
-
-	return len;
-}
-
 enum mapping_status {
 	MAPPING_ADDED,
 	MAPPING_MISSING,
@@ -611,7 +600,6 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 
 	while (copied < len) {
 		enum mapping_status status;
-		size_t discard_len = 0;
 		u32 map_remaining;
 		int bytes_read;
 		u64 ack_seq;
@@ -627,20 +615,27 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 		if (status == MAPPING_ADDED) {
 			/* Common case, but nothing to do here */
 		} else if (status == MAPPING_MISSING) {
-			if (!subflow->map_valid) {
-				pr_debug("Mapping missing, trying next skb");
-
-				arg.msg = NULL;
-				desc.count = SIZE_MAX;
-
-				bytes_read = tcp_read_sock(ssk, &desc,
-							   mptcp_flush_actor);
-
-				if (bytes_read < 0)
-					break;
+			struct sk_buff *skb = skb_peek(&ssk->sk_receive_queue);
 
+			if (!skb->len) {
+				/* the TCP stack deliver 0 len FIN pkt
+				 * to the receive queue, that is the only
+				 * 0len pkts ever expected here, and we can
+				 * admit no mapping only for 0 len pkts
+				 */
+				if (!(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN))
+					WARN_ONCE(1, "0len seq %d:%d flags %x",
+						  TCP_SKB_CB(skb)->seq,
+						  TCP_SKB_CB(skb)->end_seq,
+						  TCP_SKB_CB(skb)->tcp_flags);
+				sk_eat_skb(sk, skb);
 				continue;
 			}
+			if (!subflow->map_valid) {
+				WARN_ONCE(1, "corrupted stream: missing mapping");
+				copied = -EBADFD;
+				break;
+			}
 		} else if (status == MAPPING_EMPTY) {
 			goto wait_for_data;
 		} else if (status == MAPPING_DATA_FIN) {
@@ -658,60 +653,44 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 
 		if (unlikely(before(ssn, subflow->map_subflow_seq))) {
 			/* Mapping covers data later in the subflow stream,
-			 * discard unmapped data.
+			 * currently unsupported.
 			 */
-			pr_debug("Mapping covers data later in stream");
-			discard_len = subflow->map_subflow_seq - ssn;
+			WARN_ONCE(1, "Mapping covers data later in stream");
+			copied = -EBADFD;
+			break;
 		} else if (unlikely(!before(ssn, (subflow->map_subflow_seq +
 						  subflow->map_data_len)))) {
 			/* Mapping ends earlier in the subflow stream.
-			 * Invalidate the mapping and try again.
+			 * Invalid
 			 */
-			subflow->map_valid = 0;
-			pr_debug("Invalid mapping ssn=%d map_seq=%d map_data_len=%d",
-				 ssn, subflow->map_subflow_seq,
-				 subflow->map_data_len);
-			continue;
-		} else {
-			ack_seq = get_mapped_dsn(subflow);
-
-			if (before64(ack_seq, old_ack)) {
-				/* Mapping covers data already received,
-				 * discard data in the current mapping
-				 * and invalidate the map

net/mptcp/protocol.c static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, · line 667

Quoted 10 lines
net/mptcp/protocol.c +2 -8
-				 */
-				u64 map_end_dsn = subflow->map_seq +
-					subflow->map_data_len;
-				discard_len = min(map_end_dsn - ack_seq,
-						  old_ack - ack_seq);
-				subflow->map_valid = 0;
-				pr_debug("Duplicate MPTCP data found");
-			}
+			WARN_ONCE(1, "Mapping covers old data ssn=%d "
+				  "map_seq=%d map_data_len=%d", ssn,

Checkpatch will complain about concatenating string constants like this - it allows long lines when string constants are involved.

Quoted 24 lines
net/mptcp/protocol.c +10 -10
+				  subflow->map_subflow_seq,
+				  subflow->map_data_len);
+			copied = -EBADFD;
+			break;
 		}
 
-		if (discard_len) {
-			/* Discard data for the current mapping.
-			 */
-			pr_debug("Discard %zu bytes", discard_len);
+		ack_seq = get_mapped_dsn(subflow);
+		map_remaining = subflow->map_data_len - get_map_offset(subflow);
 
+		if (before64(ack_seq, old_ack)) {
+			/* Mapping covers data already received, discard data
+			 * in the current mapping
+			 */
 			arg.msg = NULL;
-			desc.count = discard_len;
-
-			bytes_read = tcp_read_sock(ssk, &desc,
-						   mptcp_read_actor);
-
-			if (bytes_read < 0)

net/mptcp/protocol.c static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, · line 686

Quoted 10 lines
net/mptcp/protocol.c +5 -5
-				break;
-			else if (bytes_read == discard_len)
-				continue;
-			else
-				goto wait_for_data;
+			desc.count = min_t(size_t, old_ack - ack_seq,
+					   map_remaining);
+			pr_debug("Duplicate MPTCP data found, map "
+				 "remaining %d already acked data %lld "
+				 "discarding %ld bytes", map_remaining,

Same string concatenation comment as above.

Thanks,

Mat

Quoted 21 lines
net/mptcp/protocol.c +6 -4
+				 old_ack - ack_seq, desc.count);
+		} else {
+			arg.msg = msg;
+			desc.count = min_t(size_t, len - copied, map_remaining);
 		}
 
 		/* Read mapped data */
-		map_remaining = subflow->map_data_len - get_map_offset(subflow);
-		desc.count = min_t(size_t, len - copied, map_remaining);
-		arg.msg = msg;
 		bytes_read = tcp_read_sock(ssk, &desc, mptcp_read_actor);
 		if (bytes_read < 0)
 			break;
@@ -730,7 +709,8 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 				 subflow->map_data_len);
 		}
 
-		copied += bytes_read;
+		if (arg.msg)
+			copied += bytes_read;
 
Quoted 10 lines

wait_for_data:
if (copied)

Signature · 6 lines
2.21.0

_______________________________________________
mptcp mailing list
mptcp(a)lists.01.org
https://lists.01.org/mailman/listinfo/mptcp

--
Mat Martineau
Intel

#7 lore raw

Re: [MPTCP] [PATCH 2/2] mptcp: recvmsg() should really try to deliver 'len' bytes to user-space

In reply to Paolo Abeni
Quoted 28 lines · Paolo Abeni

Currently we break the recvmsg() main loop as soon as we have copied any amount of data or the ssk is closed.

With reinjection we can receive duplicate data and reach the 'wait_for_data' label with 'copied == 0'.
The subflow sk could be already in closed status, if the local socket has already received all the data from the peer (and the data is sitting into the receive queue).

When we meet the above scenario, currently recvmsg() terminates with return code '0' and the user space interpret that as EoF, even if there is potentially a lot of data pending.

This change addresses the issue checking for pending data before going through the 'wait_for_data' code path.

Signed-off-by: Paolo Abeni <pabeni(a)redhat.com>
--
I'm ok with squashing to commit 786f07cf97f1 ("mptcp: Implement MPTCP receive path")
---
net/mptcp/protocol.c | 7 +++++++

1 file changed, 7 insertions(+)
net/mptcp/protocol.c +0 -0
index 4fa4e07df17c..402a43d5dac5 100644

net/mptcp/protocol.c static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, · line 715

Quoted 4 lines
net/mptcp/protocol.c +1 -0
@@ -712,6 +712,13 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 		if (arg.msg)
 			copied += bytes_read;
 
+		/* The 'wait_for_data' code path can terminte the receive loop

terminte->terminate

Everything else looks good to me.

Quoted 7 lines
net/mptcp/protocol.c +6 -0
+		 * in a number of scenarios: check if more data is pending
+		 * before giving up.
+		 */
+		if (!skb_queue_empty(&ssk->sk_receive_queue))
+			continue;
+

wait_for_data:

Quoted 10 lines
		if (copied)
			break;
Signature · 6 lines
2.21.0

_______________________________________________
mptcp mailing list
mptcp(a)lists.01.org
https://lists.01.org/mailman/listinfo/mptcp

--
Mat Martineau
Intel

author
#4 lore raw

Re: [MPTCP] [PATCH 1/2] mptcp: refactor recvmsg()

In reply to Mat Martineau
Quoted 35 lines · Mat Martineau
(quoted diff) +0 -0
@@ -658,60 +653,44 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
Quoted 34 lines
		if (unlikely(before(ssn, subflow->map_subflow_seq))) {
			/* Mapping covers data later in the subflow stream,
net/mptcp/protocol.c +1 -1
-			 * discard unmapped data.
+			 * currently unsupported.
			 */
net/mptcp/protocol.c +3 -2
-			pr_debug("Mapping covers data later in stream");
-			discard_len = subflow->map_subflow_seq - ssn;
+			WARN_ONCE(1, "Mapping covers data later in stream");
+			copied = -EBADFD;
+			break;
		} else if (unlikely(!before(ssn, (subflow->map_subflow_seq +
						  subflow->map_data_len)))) {
			/* Mapping ends earlier in the subflow stream.
net/mptcp/protocol.c +1 -1
-			 * Invalidate the mapping and try again.
+			 * Invalid
			 */
net/mptcp/protocol.c +0 -17
-			subflow->map_valid = 0;
-			pr_debug("Invalid mapping ssn=%d map_seq=%d map_data_len=%d",
-				 ssn, subflow->map_subflow_seq,
-				 subflow->map_data_len);
-			continue;
-		} else {
-			ack_seq = get_mapped_dsn(subflow);
-
-			if (before64(ack_seq, old_ack)) {
-				/* Mapping covers data already received,
-				 * discard data in the current mapping
-				 * and invalidate the map
-				 */
-				u64 map_end_dsn = subflow->map_seq +
-					subflow->map_data_len;
-				discard_len = min(map_end_dsn - ack_seq,
-						  old_ack - ack_seq);
Quoted 8 lines

net/mptcp/protocol.c static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, · line 667

Quoted 5 lines
net/mptcp/protocol.c +2 -3
-				subflow->map_valid = 0;
-				pr_debug("Duplicate MPTCP data found");
-			}
+			WARN_ONCE(1, "Mapping covers old data ssn=%d "
+				  "map_seq=%d map_data_len=%d", ssn,

Checkpatch will complain about concatenating string constants like this - it allows long lines when string constants are involved.

Thank you for the review.

I was unable to compress the message in a single line fitting 80 chars, so I opted for splitting it (checkpatch would complain anyway)

I'll mangle the messages and re-submit (fixing the typo in 2/2, too)

Thanks,

Paolo

#5 lore raw

Re: [MPTCP] [PATCH 1/2] mptcp: refactor recvmsg()

Quoted 45 lines · Paolo Abeni
Quoted 35 lines · Mat Martineau
(quoted diff) +0 -0
@@ -658,60 +653,44 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
Quoted 34 lines
		if (unlikely(before(ssn, subflow->map_subflow_seq))) {
			/* Mapping covers data later in the subflow stream,
net/mptcp/protocol.c +1 -1
-			 * discard unmapped data.
+			 * currently unsupported.
			 */
net/mptcp/protocol.c +3 -2
-			pr_debug("Mapping covers data later in stream");
-			discard_len = subflow->map_subflow_seq - ssn;
+			WARN_ONCE(1, "Mapping covers data later in stream");
+			copied = -EBADFD;
+			break;
		} else if (unlikely(!before(ssn, (subflow->map_subflow_seq +
						  subflow->map_data_len)))) {
			/* Mapping ends earlier in the subflow stream.
net/mptcp/protocol.c +1 -1
-			 * Invalidate the mapping and try again.
+			 * Invalid
			 */
net/mptcp/protocol.c +0 -17
-			subflow->map_valid = 0;
-			pr_debug("Invalid mapping ssn=%d map_seq=%d map_data_len=%d",
-				 ssn, subflow->map_subflow_seq,
-				 subflow->map_data_len);
-			continue;
-		} else {
-			ack_seq = get_mapped_dsn(subflow);
-
-			if (before64(ack_seq, old_ack)) {
-				/* Mapping covers data already received,
-				 * discard data in the current mapping
-				 * and invalidate the map
-				 */
-				u64 map_end_dsn = subflow->map_seq +
-					subflow->map_data_len;
-				discard_len = min(map_end_dsn - ack_seq,
-						  old_ack - ack_seq);
Quoted 8 lines

net/mptcp/protocol.c static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, · line 667

Quoted 5 lines
net/mptcp/protocol.c +2 -3
-				subflow->map_valid = 0;
-				pr_debug("Duplicate MPTCP data found");
-			}
+			WARN_ONCE(1, "Mapping covers old data ssn=%d "
+				  "map_seq=%d map_data_len=%d", ssn,

Checkpatch will complain about concatenating string constants like this - it allows long lines when string constants are involved.

Quoted 4 lines

Thank you for the review.

I was unable to compress the message in a single line fitting 80 chars, so I opted for splitting it (checkpatch would complain anyway)

That's the obscure detail I was trying to get at: checkpatch does not complain about lines over 80 chars when the length is due to a long string constant. It is ok to have long lines in that case, so there's no need to compress the messages.

Quoted 2 lines

I'll mangle the messages and re-submit (fixing the typo in 2/2, too)

Thanks for the fixes!

--
Mat Martineau
Intel