Skip to contentSkip to conversation

mptcp: implement retransmit infrastructure

PATCH In review Paolo Abeni 2 participants 11 messages Aug 30, 2019 – Sep 3, 2019

Subsystems (2)
lore
Share & tools
lore

Paolo Abeni

author
#0 lore raw

cover mptcp: implement retransmit infrastructure

This is the first submission aming at git repo inclusion. It includes several clean-ups, and covers feedback from Mat about una_seq updates and 32 bit expansion.

In my testbet it survived a good number of self-tests iteration, but I can't exclude more bugs are pending.

Please see the individual patches for the detailed changelog

RFC v2 -> v1
- update msk una_seq in tcp_incoming_options() and drop old patch 3/8
- cleanup timer management
- cleanup dfrag management

Paolo Abeni (7):
mptcp: move before/after64 into the header file
mptcp: update per unacked sequence on pkt reception
mptcp: queue data for mptcp level retransmission
mptcp: introduce MPTCP retransmission timer
mptcp: implement memory accounting for mptcp rtx queue
mptcp: rework mptcp_sendmsg_frag to accept optional dfrag
mptcp: implement and use MPTCP-level retransmission

 net/mptcp/options.c  |  40 ++++-
 net/mptcp/protocol.c | 395 +++++++++++++++++++++++++++++++++++++++----
 net/mptcp/protocol.h |  41 +++++
 3 files changed, 439 insertions(+), 37 deletions(-)
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH 1/7 mptcp: move before/after64 into the header file

So that we can reuse in multiple files

Signed-off-by: Paolo Abeni <pabeni(a)redhat.com>
--
Squash to "mptcp: Implement MPTCP receive path"

---
 net/mptcp/protocol.c | 7 -------
 net/mptcp/protocol.h | 7 +++++++
 2 files changed, 7 insertions(+), 7 deletions(-)
net/mptcp/protocol.c +0 -7
index c082a2731a66..15bd54f43d4b 100644
@@ -17,13 +17,6 @@
 #include <net/mptcp.h>
 #include "protocol.h"
 
-static inline bool before64(__u64 seq1, __u64 seq2)
-{
-	return (__s64)(seq1 - seq2) < 0;
-}
-
-#define after64(seq2, seq1)	before64(seq1, seq2)
-
 static struct socket *__mptcp_fallback_get_ref(const struct mptcp_sock *msk)
 {
 	sock_owned_by_me((const struct sock *)msk);
net/mptcp/protocol.h +7 -0
index 991ad288b892..8c36a379f67d 100644
@@ -279,4 +279,11 @@ static inline struct mptcp_ext *mptcp_get_ext(struct sk_buff *skb)
 	return (struct mptcp_ext *)skb_ext_find(skb, SKB_EXT_MPTCP);
 }
 
+static inline bool before64(__u64 seq1, __u64 seq2)
+{
+	return (__s64)(seq1 - seq2) < 0;
+}
+
+#define after64(seq2, seq1)	before64(seq1, seq2)
+
 #endif /* __MPTCP_PROTOCOL_H */
Signature · 1 line
2.21.0
author
#2 lore raw

PATCH 2/7 mptcp: update per unacked sequence on pkt reception

In reply to Paolo Abeni

So that we keep per unacked sequence number consistent; since we update per msk data, use an atomic64 cmpxcgh() to protect against concurrent updates from multiple subflows.

Initialize the snd_una at connect()/accept() time.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
rfc v2 -> v1:
- do msk-level accounting, using atomic64_t

rfc v1 -> rfc v2:
- initialize subflow snd_una on subflow connect()/accept()

---
 net/mptcp/options.c  | 39 +++++++++++++++++++++++++++++++++------
 net/mptcp/protocol.c |  2 ++
 net/mptcp/protocol.h |  1 +
 3 files changed, 36 insertions(+), 6 deletions(-)
net/mptcp/options.c +33 -6
index be7afa5aac6a..854dc050c1da 100644
@@ -524,6 +524,26 @@ bool mptcp_synack_options(const struct request_sock *req, unsigned int *size,
 	return false;
 }
 
+static u64 expand_ack(u64 old_ack, u64 cur_ack, bool use_64bit)
+{
+	u32 old_ack32, cur_ack32;
+
+	if (use_64bit)
+		return cur_ack;
+
+	old_ack32 = (u32)old_ack;
+	cur_ack32 = (u32)cur_ack;
+	cur_ack = (old_ack & GENMASK_ULL(63, 32)) + cur_ack32;
+	if (unlikely(before(cur_ack32, old_ack32)))
+		return cur_ack + (1LL << 32);
+	return cur_ack;
+}
+
+void update_una(struct mptcp_sock *msk, u64 old_una, u64 new_una)
+{
+	atomic64_cmpxchg(&msk->snd_una, old_una, new_una);
+}
+
 void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 			    struct tcp_options_received *opt_rx)
 {
@@ -547,6 +567,19 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 	if (!mp_opt->dss)
 		return;
 
+	/* we can't wait for recvmsg() to update the ack_seq, otherwise
+	 * monodirectional flows will stuck
+	 */
+	if (msk && mp_opt->use_ack) {
+		u64 snd_una = atomic64_read(&msk->snd_una);
+		u64 new_snd_una;
+
+		new_snd_una = expand_ack(snd_una, mp_opt->data_ack,
+					 mp_opt->ack64);
+		if (after64(new_snd_una, snd_una))
+			update_una(msk, snd_una, new_snd_una);
+	}
+
 	mpext = skb_ext_add(skb, SKB_EXT_MPTCP);
 	if (!mpext)
 		return;
@@ -563,12 +596,6 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 		mpext->use_checksum = mp_opt->use_checksum;
 	}
 
-	if (mp_opt->use_ack) {
-		mpext->data_ack = mp_opt->data_ack;
-		mpext->use_ack = 1;
-		mpext->ack64 = mp_opt->ack64;
-	}
-
 	mpext->data_fin = mp_opt->data_fin;
 
 	if (msk)
net/mptcp/protocol.c +2 -0
index 15bd54f43d4b..52e39fbb217d 100644
@@ -696,6 +696,7 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,
 
 		crypto_key_sha1(msk->remote_key, NULL, &ack_seq);
 		msk->write_seq = subflow->idsn + 1;
+		atomic64_set(&msk->snd_una, msk->write_seq);
 		ack_seq++;
 		msk->ack_seq = ack_seq;
 		subflow->map_seq = ack_seq;
@@ -833,6 +834,7 @@ void mptcp_finish_connect(struct sock *sk, int mp_capable)
 
 		crypto_key_sha1(msk->remote_key, NULL, &ack_seq);
 		msk->write_seq = subflow->idsn + 1;
+		atomic64_set(&msk->snd_una, msk->write_seq);
 		ack_seq++;
 		msk->ack_seq = ack_seq;
 		subflow->map_seq = ack_seq;
net/mptcp/protocol.h +1 -0
index 8c36a379f67d..e7c29007f0fe 100644
@@ -119,6 +119,7 @@ struct mptcp_sock {
 	u64		remote_key;
 	u64		write_seq;
 	u64		ack_seq;
+	atomic64_t	snd_una;
 	u32		token;
 	unsigned long	flags;
 	u16		dport;
Signature · 1 line
2.21.0
author
#6 lore raw

PATCH 3/7 mptcp: queue data for mptcp level retransmission

In reply to Paolo Abeni

keep the send page fragment on an MPTCP level retransmission queue. the queue entries are allocated inside the page frag allocator, acquiring an additional reference to the page for each list entry.

Also switch to a custom page frag refill function, to ensure that the current page fragment can always host an MPTCP rtx queue entry.

The MPTCP rtx queue is flushed at disconnect() and close() time

Note that now we need to call __mptcp_init_sock() regarless of mptcp enable status, as the destructor will try to walk the rtx_queue.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
rfc v2 -> v1:
- clear rtx queue even before sendmsg
- intrduce an helper to clear a single data frag

rfc v1 -> rfc v2:
- rtx_queue_tail is now static inline (fix link issue)
- simpler ALIGN() usage (Florian)
- fix pfrag accounting, adding rtx entry overhead when allocating a new one
- fixed page -> virt conversion
- always init mptcp data structs
- move rtx queue flushing into both disconnect() and close()

---
 net/mptcp/protocol.c | 131 ++++++++++++++++++++++++++++++++++++++++---
 net/mptcp/protocol.h |  20 +++++++
 2 files changed, 143 insertions(+), 8 deletions(-)
net/mptcp/protocol.c +123 -8
index 52e39fbb217d..2f5125961333 100644
@@ -67,14 +67,74 @@ static inline bool mptcp_skb_can_collapse_to(const struct mptcp_sock *msk,
 	return mpext && mpext->data_seq + mpext->data_len == msk->write_seq;
 }
 
+static inline bool mptcp_frag_can_collapse_to(const struct mptcp_sock *msk,
+					      const struct page_frag *pfrag,
+					      const struct mptcp_data_frag *df)
+{
+	return df && pfrag->page == df->page &&
+		df->data_seq + df->data_len == msk->write_seq;
+}
+
+static void dfrag_clear(struct mptcp_data_frag *dfrag)
+{
+	list_del(&dfrag->list);
+	put_page(dfrag->page);
+}
+
+static void mptcp_clean_una(struct sock *sk)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct mptcp_data_frag *dtmp, *dfrag;
+	u64 snd_una = atomic64_read(&msk->snd_una);
+
+	list_for_each_entry_safe(dfrag, dtmp, &msk->rtx_queue, list) {
+		if (after64(dfrag->data_seq + dfrag->data_len, snd_una))
+			break;
+
+		dfrag_clear(dfrag);
+	}
+}
+
+/* ensure we get enough memory for the frag hdr, beyond some minimal amount of
+ * data
+ */
+bool mptcp_page_frag_refill(struct sock *sk, struct page_frag *pfrag)
+{
+	if (likely(skb_page_frag_refill(32U + sizeof(struct mptcp_data_frag),
+					pfrag, sk->sk_allocation)))
+		return true;
+
+	sk->sk_prot->enter_memory_pressure(sk);
+	sk_stream_moderate_sndbuf(sk);
+	return false;
+}
+
+static inline struct mptcp_data_frag *
+mptcp_carve_data_frag(const struct mptcp_sock *msk, struct page_frag *pfrag,
+		      int orig_offset)
+{
+	int offset = ALIGN(orig_offset, sizeof(long));
+	struct mptcp_data_frag *dfrag;
+
+	dfrag = (struct mptcp_data_frag *)(page_to_virt(pfrag->page) + offset);
+	dfrag->data_len = 0;
+	dfrag->data_seq = msk->write_seq;
+	dfrag->overhead = offset - orig_offset + sizeof(struct mptcp_data_frag);
+	dfrag->offset = offset + sizeof(struct mptcp_data_frag);
+	dfrag->page = pfrag->page;
+
+	return dfrag;
+}
+
 static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 			      struct msghdr *msg, long *timeo, int *pmss_now,
 			      int *ps_goal)
 {
-	int mss_now, avail_size, size_goal, ret;
+	int mss_now, avail_size, size_goal, offset, ret, frag_truesize = 0;
+	bool dfrag_collapsed, collapsed, can_collapse = false;
 	struct mptcp_sock *msk = mptcp_sk(sk);
-	bool collapsed, can_collapse = false;
 	struct mptcp_ext *mpext = NULL;
+	struct mptcp_data_frag *dfrag;
 	struct page_frag *pfrag;
 	struct sk_buff *skb;
 	size_t psize;
@@ -83,10 +143,15 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	 * from one substream to another, but do per subflow memory accounting
 	 */
 	pfrag = sk_page_frag(sk);
-	while (!sk_page_frag_refill(ssk, pfrag)) {
+	while (!mptcp_page_frag_refill(ssk, pfrag)) {
 		ret = sk_stream_wait_memory(ssk, timeo);
 		if (ret)
 			return ret;
+
+		/* id sk_stream_wait_memory() sleeps snd_una can change
+		 * significantly, refresh the rtx queue
+		 */
+		mptcp_clean_una(sk);
 	}
 
 	/* compute copy limit */
@@ -113,11 +178,23 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 		else
 			can_collapse = false;
 	}
-	psize = min_t(size_t, pfrag->size - pfrag->offset, avail_size);
+
+	/* reuse tail pfrag, if possible, or carve a new one from the page
+	 * allocator
+	 */
+	dfrag = mptcp_rtx_tail(sk);
+	offset = pfrag->offset;
+	dfrag_collapsed = mptcp_frag_can_collapse_to(msk, pfrag, dfrag);
+	if (!dfrag_collapsed) {
+		dfrag = mptcp_carve_data_frag(msk, pfrag, offset);
+		offset = dfrag->offset;
+		frag_truesize = dfrag->overhead;
+	}
+	psize = min_t(size_t, pfrag->size - offset, avail_size);
 
 	/* Copy to page */
 	pr_debug("left=%zu", msg_data_left(msg));
-	psize = copy_page_from_iter(pfrag->page, pfrag->offset,
+	psize = copy_page_from_iter(pfrag->page, offset,
 				    min_t(size_t, msg_data_left(msg), psize),
 				    &msg->msg_iter);
 	pr_debug("left=%zu", msg_data_left(msg));
@@ -127,13 +204,24 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	/* tell the TCP stack to delay the push so that we can safely
 	 * access the skb after the sendpages call
 	 */
-	ret = do_tcp_sendpages(ssk, pfrag->page, pfrag->offset, psize,
+	ret = do_tcp_sendpages(ssk, pfrag->page, offset, psize,
 			       msg->msg_flags | MSG_SENDPAGE_NOTLAST);
 	if (ret <= 0)
 		return ret;
+
+	frag_truesize += ret;
 	if (unlikely(ret < psize))
 		iov_iter_revert(&msg->msg_iter, psize - ret);
 
+	/* send successful, keep track of sent data for mptcp-level
+	 * retransmission
+	 */
+	dfrag->data_len += ret;
+	if (!dfrag_collapsed) {
+		get_page(dfrag->page);
+		list_add_tail(&dfrag->list, &msk->rtx_queue);
+	}
+
 	collapsed = skb == tcp_write_queue_tail(ssk);
 	BUG_ON(collapsed && !can_collapse);
 	if (collapsed) {
@@ -162,7 +250,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	 */
 
 out:
-	pfrag->offset += ret;
+	pfrag->offset += frag_truesize;
 	msk->write_seq += ret;
 	subflow_ctx(ssk)->rel_write_seq += ret;
 
@@ -209,6 +297,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	}
 
 	lock_sock(ssk);
+	mptcp_clean_una(sk);
 	timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
 	while (msg_data_left(msg)) {
 		ret = mptcp_sendmsg_frag(sk, ssk, msg, &timeo, &mss_now,
@@ -608,16 +697,31 @@ static int __mptcp_init_sock(struct sock *sk)
 	pr_debug("msk=%p", msk);
 
 	INIT_LIST_HEAD(&msk->conn_list);
+	INIT_LIST_HEAD(&msk->rtx_queue);
 
 	return 0;
 }
 
 static int mptcp_init_sock(struct sock *sk)
 {
+	int ret = __mptcp_init_sock(sk);
+
+	if (ret)
+		return ret;
+
 	if (!mptcp_is_enabled(sock_net(sk)))
 		return -ENOPROTOOPT;
 
-	return __mptcp_init_sock(sk);
+	return 0;
+}
+
+static void __mptcp_clear_xmit(struct sock *sk)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct mptcp_data_frag *dtmp, *dfrag;
+
+	list_for_each_entry_safe(dfrag, dtmp, &msk->rtx_queue, list)
+		dfrag_clear(dfrag);
 }
 
 static void mptcp_close(struct sock *sk, long timeout)
@@ -644,11 +748,21 @@ static void mptcp_close(struct sock *sk, long timeout)
 		pr_debug("conn_list->subflow=%p", subflow);
 		sock_release(mptcp_subflow_tcp_socket(subflow));
 	}
+
+	__mptcp_clear_xmit(sk);
 	release_sock(sk);
 
 	sk_common_release(sk);
 }
 
+static int mptcp_disconnect(struct sock *sk, int flags)
+{
+	lock_sock(sk);
+	__mptcp_clear_xmit(sk);
+	release_sock(sk);
+	return tcp_disconnect(sk, flags);
+}
+
 static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,
 				 bool kern)
 {
@@ -875,6 +989,7 @@ static struct proto mptcp_prot = {
 	.name		= "MPTCP",
 	.owner		= THIS_MODULE,
 	.init		= mptcp_init_sock,
+	.disconnect	= mptcp_disconnect,
 	.close		= mptcp_close,
 	.accept		= mptcp_accept,
 	.setsockopt	= mptcp_setsockopt,
net/mptcp/protocol.h +20 -0
index e7c29007f0fe..2d19d534ff9f 100644
@@ -111,6 +111,15 @@ struct mptcp_pm_data {
 	u32	token;
 };
 
+struct mptcp_data_frag {
+	struct list_head list;
+	u64 data_seq;
+	int data_len;
+	int offset;
+	int overhead;
+	struct page *page;
+};
+
 /* MPTCP connection sock */
 struct mptcp_sock {
 	/* inet_connection_sock must be the first member */
@@ -124,6 +133,7 @@ struct mptcp_sock {
 	unsigned long	flags;
 	u16		dport;
 	struct list_head conn_list;
+	struct list_head rtx_queue;
 	struct socket	*subflow; /* outgoing connect/listener/!mp_capable */
 	struct mptcp_pm_data	pm;
 	u8		addr_signal;
@@ -137,6 +147,16 @@ static inline struct mptcp_sock *mptcp_sk(const struct sock *sk)
 	return (struct mptcp_sock *)sk;
 }
 
+static inline struct mptcp_data_frag *mptcp_rtx_tail(const struct sock *sk)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
+	if (list_empty(&msk->rtx_queue))
+		return NULL;
+
+	return list_last_entry(&msk->rtx_queue, struct mptcp_data_frag, list);
+}
+
 struct subflow_request_sock {
 	struct	tcp_request_sock sk;
 	u8	mp_capable : 1,
Signature · 1 line
2.21.0
author
#7 lore raw

PATCH 4/7 mptcp: introduce MPTCP retransmission timer

In reply to Paolo Abeni

The timer will be used to schedule retransmission. It's frequency is based on the current subflow RTO estimation and is reset on every una_seq update

The timer is clearer for good by __mptcp_clear_xmit()

Also clean MPTCP rtx queue before each transmission

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
rfc v2 -> v1:
- don't update anymore una_seq inside the timer
- reset timer from mptcp_incoming_options() context
- update and store the timer interval at each sendmsg() call
- added new timer pending helper
- plug refcnt leak in mptcp_retransmit_timer()
- really use the mptcp release cb

rfc v1 -> rfc v2:
- protect against wrap-around while cleaning-up dfrags in mptcp_update_msk_una()

---
 net/mptcp/options.c  |   3 +-
 net/mptcp/protocol.c | 100 +++++++++++++++++++++++++++++++++++++++++++
 net/mptcp/protocol.h |   2 +
 3 files changed, 104 insertions(+), 1 deletion(-)
net/mptcp/options.c +2 -1
index 854dc050c1da..ebb9d3dd7a6a 100644
@@ -541,7 +541,8 @@ static u64 expand_ack(u64 old_ack, u64 cur_ack, bool use_64bit)
 
 void update_una(struct mptcp_sock *msk, u64 old_una, u64 new_una)
 {
-	atomic64_cmpxchg(&msk->snd_una, old_una, new_una);
+	if (atomic64_cmpxchg(&msk->snd_una, old_una, new_una) == old_una)
+		mptcp_reset_timer((struct sock *)msk);
 }
 
 void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
net/mptcp/protocol.c +100 -0

146 lines not on this page · raw message

net/mptcp/protocol.h +2 -0
index 2d19d534ff9f..a14fdc161813 100644
@@ -129,6 +129,7 @@ struct mptcp_sock {
 	u64		write_seq;
 	u64		ack_seq;
 	atomic64_t	snd_una;
+	unsigned long	timer_ival;
 	u32		token;
 	unsigned long	flags;
 	u16		dport;
@@ -249,6 +250,7 @@ void mptcp_get_options(const struct sk_buff *skb,
 
 void mptcp_finish_connect(struct sock *sk, int mp_capable);
 void mptcp_finish_join(struct sock *sk);
+void mptcp_reset_timer(struct sock *sk);
 
 void token_init(void);
 void token_new_request(struct request_sock *req, const struct sk_buff *skb);
Signature · 1 line
2.21.0
author
#8 lore raw

PATCH 5/7 mptcp: implement memory accounting for mptcp rtx queue

In reply to Paolo Abeni

Charge the data on the rtx queue to the master MPTCP socket, too. Such memory in uncharged when the data is acked/dequeued.

Also account mptcp sockets inuse via a protocol specific pcpu counter.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
rfc v2 -> v1:
- plug completely with tcp memory accounting including memory pressure.
- use mptcp-specific socket in use counter

rfc v1 -> rfc v2:
- initialize memory_allocated/sysctl_mem fields in struct proto, required to have mem accounting primitives do their work
- deal with mem scheduling failure, bailing out the xmit

---
 net/mptcp/protocol.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +27 -3
index 2e294e043c0b..fe5f28f74422 100644
@@ -17,6 +17,8 @@
 #include <net/mptcp.h>
 #include "protocol.h"
 
+static struct percpu_counter mptcp_sockets_allocated;
+
 static void mptcp_set_timeout(const struct sock *sk, const struct sock *ssk)
 {
 	unsigned long tout = ssk && inet_csk(ssk)->icsk_pending ?
@@ -110,9 +112,10 @@ static inline bool mptcp_frag_can_collapse_to(const struct mptcp_sock *msk,
 		df->data_seq + df->data_len == msk->write_seq;
 }
 
-static void dfrag_clear(struct mptcp_data_frag *dfrag)
+static void dfrag_clear(struct sock *sk, struct mptcp_data_frag *dfrag)
 {
 	list_del(&dfrag->list);
+	sk_mem_uncharge(sk, dfrag->data_len + dfrag->overhead);
 	put_page(dfrag->page);
 }
 
@@ -126,8 +129,9 @@ static void mptcp_clean_una(struct sock *sk)
 		if (after64(dfrag->data_seq + dfrag->data_len, snd_una))
 			break;
 
-		dfrag_clear(dfrag);
+		dfrag_clear(sk, dfrag);
 	}
+	sk_mem_reclaim_partial(sk);
 }
 
 /* ensure we get enough memory for the frag hdr, beyond some minimal amount of
@@ -236,6 +240,9 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	if (!psize)
 		return -EINVAL;
 
+	if (!sk_wmem_schedule(sk, psize + dfrag->overhead))
+		return -ENOMEM;
+
 	/* tell the TCP stack to delay the push so that we can safely
 	 * access the skb after the sendpages call
 	 */
@@ -257,6 +264,11 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 		list_add_tail(&dfrag->list, &msk->rtx_queue);
 	}
 
+	/* charge data on mptcp rtx queue to the master socket
+	 * Note: we charge such data both to sk and ssk
+	 */
+	sk->sk_forward_alloc -= frag_truesize;
+
 	collapsed = skb == tcp_write_queue_tail(ssk);
 	BUG_ON(collapsed && !can_collapse);
 	if (collapsed) {
@@ -782,6 +794,8 @@ static int mptcp_init_sock(struct sock *sk)
 	if (ret)
 		return ret;
 
+	sk_sockets_allocated_inc(sk);
+
 	if (!mptcp_is_enabled(sock_net(sk)))
 		return -ENOPROTOOPT;
 
@@ -796,7 +810,7 @@ static void __mptcp_clear_xmit(struct sock *sk)
 	sk_stop_timer(sk, &msk->sk.icsk_retransmit_timer);
 
 	list_for_each_entry_safe(dfrag, dtmp, &msk->rtx_queue, list)
-		dfrag_clear(dfrag);
+		dfrag_clear(sk, dfrag);
 }
 
 static void mptcp_close(struct sock *sk, long timeout)
@@ -914,6 +928,8 @@ static void mptcp_destroy(struct sock *sk)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
 
+	sk_sockets_allocated_dec(sk);
+
 	pr_debug("msk=%p, subflow=%p", sk, msk->subflow->sk);
 
 	token_destroy(msk->token);
@@ -1101,6 +1117,11 @@ static struct proto mptcp_prot = {
 	.hash		= inet_hash,
 	.unhash		= inet_unhash,
 	.get_port	= mptcp_get_port,
+	.sockets_allocated	= &mptcp_sockets_allocated,
+	.memory_allocated	= &tcp_memory_allocated,
+	.memory_pressure	= &tcp_memory_pressure,
+	.sysctl_wmem_offset	= offsetof(struct net, ipv4.sysctl_tcp_wmem),
+	.sysctl_mem	= sysctl_tcp_mem,
 	.obj_size	= sizeof(struct mptcp_sock),
 	.no_autobind	= 1,
 };
@@ -1343,6 +1364,9 @@ void mptcp_proto_init(void)
 	mptcp_stream_ops.listen = mptcp_listen;
 	mptcp_stream_ops.shutdown = mptcp_shutdown;
 
+	if (percpu_counter_init(&mptcp_sockets_allocated, 0, GFP_KERNEL))
+		panic("Failed to allocate MPTCP pcpu counter\n");
+
 	token_init();
 	subflow_init();
 	pm_init();
Signature · 1 line
2.21.0
author
#9 lore raw

PATCH 6/7 mptcp: rework mptcp_sendmsg_frag to accept optional dfrag

In reply to Paolo Abeni

This will simplify mptcp-level retransmission implementation in the next patch. If dfrag is provided by the caller, skip kernel space memory allocation and use data and metadata provided by the dfrag itself.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/protocol.c | 134 +++++++++++++++++++++++++------------------
 1 file changed, 79 insertions(+), 55 deletions(-)
net/mptcp/protocol.c +79 -55
index fe5f28f74422..dd671c62cf3e 100644
@@ -93,7 +93,7 @@ static struct sock *mptcp_subflow_get_ref(const struct mptcp_sock *msk)
 	return NULL;
 }
 
-static inline bool mptcp_skb_can_collapse_to(const struct mptcp_sock *msk,
+static inline bool mptcp_skb_can_collapse_to(u64 write_seq,
 					     const struct sk_buff *skb,
 					     const struct mptcp_ext *mpext)
 {
@@ -101,7 +101,7 @@ static inline bool mptcp_skb_can_collapse_to(const struct mptcp_sock *msk,
 		return false;
 
 	/* can collapse only if MPTCP level sequence is in order */
-	return mpext && mpext->data_seq + mpext->data_len == msk->write_seq;
+	return mpext && mpext->data_seq + mpext->data_len == write_seq;
 }
 
 static inline bool mptcp_frag_can_collapse_to(const struct mptcp_sock *msk,
@@ -166,31 +166,44 @@ mptcp_carve_data_frag(const struct mptcp_sock *msk, struct page_frag *pfrag,
 }
 
 static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
-			      struct msghdr *msg, long *timeo, int *pmss_now,
+			      struct msghdr *msg, struct mptcp_data_frag *dfrag,
+			      long *timeo, int *pmss_now,
 			      int *ps_goal)
 {
 	int mss_now, avail_size, size_goal, offset, ret, frag_truesize = 0;
 	bool dfrag_collapsed, collapsed, can_collapse = false;
 	struct mptcp_sock *msk = mptcp_sk(sk);
 	struct mptcp_ext *mpext = NULL;
-	struct mptcp_data_frag *dfrag;
+	bool retransmission = !!dfrag;
 	struct page_frag *pfrag;
 	struct sk_buff *skb;
+	struct page *page;
+	u64 *write_seq;
 	size_t psize;
+	int *poffset;
 
 	/* use the mptcp page cache so that we can easily move the data
 	 * from one substream to another, but do per subflow memory accounting
+	 * Note: pfrag is used only !retransmission, but the compiler if
+	 * fooled into a warning if we don't init here
 	 */
 	pfrag = sk_page_frag(sk);
-	while (!mptcp_page_frag_refill(ssk, pfrag)) {
-		ret = sk_stream_wait_memory(ssk, timeo);
-		if (ret)
-			return ret;
-
-		/* id sk_stream_wait_memory() sleeps snd_una can change
-		 * significantly, refresh the rtx queue
-		 */
-		mptcp_clean_una(sk);
+	if (!retransmission) {
+		while (!mptcp_page_frag_refill(ssk, pfrag)) {
+			ret = sk_stream_wait_memory(ssk, timeo);
+			if (ret)
+				return ret;
+
+			/* id sk_stream_wait_memory() sleeps snd_una can change
+			 * significantly, refresh the rtx queue
+			 */
+			mptcp_clean_una(sk);
+		}
+		write_seq = &msk->write_seq;
+		page = pfrag->page;
+	} else {
+		write_seq = &dfrag->data_seq;
+		page = dfrag->page;
 	}
 
 	/* compute copy limit */
@@ -209,7 +222,8 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 		 * is collapsing was allowed: collapsing is effectively
 		 * disabled.
 		 */
-		can_collapse = mptcp_skb_can_collapse_to(msk, skb, mpext);
+		can_collapse = mptcp_skb_can_collapse_to(*write_seq, skb,
+							 mpext);
 		if (!can_collapse)
 			TCP_SKB_CB(skb)->eor = 1;
 		else if (size_goal - skb->len > 0)
@@ -218,56 +232,66 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 			can_collapse = false;
 	}
 
-	/* reuse tail pfrag, if possible, or carve a new one from the page
-	 * allocator
-	 */
-	dfrag = mptcp_rtx_tail(sk);
-	offset = pfrag->offset;
-	dfrag_collapsed = mptcp_frag_can_collapse_to(msk, pfrag, dfrag);
-	if (!dfrag_collapsed) {
-		dfrag = mptcp_carve_data_frag(msk, pfrag, offset);
+	if (!retransmission) {
+		/* reuse tail pfrag, if possible, or carve a new one from the
+		 * page allocator
+		 */
+		dfrag = mptcp_rtx_tail(sk);
+		offset = pfrag->offset;
+		dfrag_collapsed = mptcp_frag_can_collapse_to(msk, pfrag, dfrag);
+		if (!dfrag_collapsed) {
+			dfrag = mptcp_carve_data_frag(msk, pfrag, offset);
+			offset = dfrag->offset;
+			frag_truesize = dfrag->overhead;
+		}
+		poffset = &pfrag->offset;
+		psize = min_t(size_t, pfrag->size - offset, avail_size);
+
+		/* Copy to page */
+		pr_debug("left=%zu", msg_data_left(msg));
+		psize = copy_page_from_iter(pfrag->page, offset,
+					    min_t(size_t, msg_data_left(msg),
+						  psize),
+					    &msg->msg_iter);
+		pr_debug("left=%zu", msg_data_left(msg));
+		if (!psize)
+			return -EINVAL;
+
+		if (!sk_wmem_schedule(sk, psize + dfrag->overhead))
+			return -ENOMEM;
+	} else {
 		offset = dfrag->offset;
-		frag_truesize = dfrag->overhead;
+		poffset = &dfrag->offset;
+		psize = min_t(size_t, dfrag->data_len, avail_size);
 	}
-	psize = min_t(size_t, pfrag->size - offset, avail_size);
-
-	/* Copy to page */
-	pr_debug("left=%zu", msg_data_left(msg));
-	psize = copy_page_from_iter(pfrag->page, offset,
-				    min_t(size_t, msg_data_left(msg), psize),
-				    &msg->msg_iter);
-	pr_debug("left=%zu", msg_data_left(msg));
-	if (!psize)
-		return -EINVAL;
-
-	if (!sk_wmem_schedule(sk, psize + dfrag->overhead))
-		return -ENOMEM;
 
 	/* tell the TCP stack to delay the push so that we can safely
 	 * access the skb after the sendpages call
 	 */
-	ret = do_tcp_sendpages(ssk, pfrag->page, offset, psize,
+	ret = do_tcp_sendpages(ssk, page, offset, psize,
 			       msg->msg_flags | MSG_SENDPAGE_NOTLAST);
 	if (ret <= 0)
 		return ret;
 
 	frag_truesize += ret;
-	if (unlikely(ret < psize))
-		iov_iter_revert(&msg->msg_iter, psize - ret);
+	if (!retransmission) {
+		if (unlikely(ret < psize))
+			iov_iter_revert(&msg->msg_iter, psize - ret);
 
-	/* send successful, keep track of sent data for mptcp-level
-	 * retransmission
-	 */
-	dfrag->data_len += ret;
-	if (!dfrag_collapsed) {
-		get_page(dfrag->page);
-		list_add_tail(&dfrag->list, &msk->rtx_queue);
-	}
+		/* send successful, keep track of sent data for mptcp-level
+		 * retransmission
+		 */
+		dfrag->data_len += ret;
+		if (!dfrag_collapsed) {
+			get_page(dfrag->page);
+			list_add_tail(&dfrag->list, &msk->rtx_queue);
+		}
 
-	/* charge data on mptcp rtx queue to the master socket
-	 * Note: we charge such data both to sk and ssk
-	 */
-	sk->sk_forward_alloc -= frag_truesize;
+		/* charge data on mptcp rtx queue to the master socket
+		 * Note: we charge such data both to sk and ssk
+		 */
+		sk->sk_forward_alloc -= frag_truesize;
+	}
 
 	collapsed = skb == tcp_write_queue_tail(ssk);
 	BUG_ON(collapsed && !can_collapse);
@@ -281,7 +305,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	mpext = skb_ext_add(skb, SKB_EXT_MPTCP);
 	if (mpext) {
 		memset(mpext, 0, sizeof(*mpext));
-		mpext->data_seq = msk->write_seq;
+		mpext->data_seq = *write_seq;
 		mpext->subflow_seq = subflow_ctx(ssk)->rel_write_seq;
 		mpext->data_len = ret;
 		mpext->checksum = 0xbeef;
@@ -297,8 +321,8 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	 */
 
 out:
-	pfrag->offset += frag_truesize;
-	msk->write_seq += ret;
+	*poffset += frag_truesize;
+	*write_seq += ret;
 	subflow_ctx(ssk)->rel_write_seq += ret;
 
 	return ret;
@@ -347,7 +371,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	mptcp_clean_una(sk);
 	timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
 	while (msg_data_left(msg)) {
-		ret = mptcp_sendmsg_frag(sk, ssk, msg, &timeo, &mss_now,
+		ret = mptcp_sendmsg_frag(sk, ssk, msg, NULL, &timeo, &mss_now,
 					 &size_goal);
 		if (ret < 0)
 			break;
Signature · 1 line
2.21.0
author
#10 lore raw

PATCH 7/7 mptcp: implement and use MPTCP-level retransmission

In reply to Paolo Abeni

On timeout event, schedule a work queue to do the retransmission. Retransmission code resemple closely sendmsg() implementation and re-uses mptcp_sendmsg_frag, providing a dummy msghdr - for flags' sake - and peeking the relevant dfrag from the rtx head.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
rfc v2 -> v1:
- really preserve dfrag seq across retransmission using u64 store instead of int
- update timer management according to new schema
- fix issues in mptcp_retransmit() cleanup code
- do cleanup the rtx queue before starting the retransmission
- cancel the rtx work outside the socket lock (avoid deadlock)

rfc v1 -> rfc v2:
- rtx_queue_head is now static inline (fix link issue)
- cancel on mptcp_clear_xmit
- (hopefully) fix msk refcnt

---
 net/mptcp/protocol.c | 81 ++++++++++++++++++++++++++++++++++++++++++--
 net/mptcp/protocol.h | 11 ++++++
 2 files changed, 89 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +78 -3
index dd671c62cf3e..4ab72e6fd748 100644
@@ -771,10 +771,12 @@ static void mptcp_retransmit_handler(struct sock *sk)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
 
-	if (atomic64_read(&msk->snd_una) == msk->write_seq)
+	if (atomic64_read(&msk->snd_una) == msk->write_seq) {
 		mptcp_stop_timer(sk);
-	else
-		mptcp_reset_timer(sk);
+	} else {
+		if (schedule_work(&msk->rtx_work))
+			sock_hold(sk);
+	}
 }
 
 static void mptcp_retransmit_timer(struct timer_list *t)
@@ -796,6 +798,66 @@ static void mptcp_retransmit_timer(struct timer_list *t)
 	sock_put(sk);
 }
 
+static void mptcp_retransmit(struct work_struct *work)
+{
+	int orig_len, orig_offset, ret, mss_now = 0, size_goal = 0;
+	struct mptcp_data_frag *dfrag;
+	struct sock *ssk, *sk;
+	struct mptcp_sock *msk;
+	u64 orig_write_seq;
+	size_t copied = 0;
+	struct msghdr msg;
+	long timeo = 0;
+
+	msk = container_of(work, struct mptcp_sock, rtx_work);
+	sk = &msk->sk.icsk_inet.sk;
+
+	lock_sock(sk);
+	mptcp_clean_una(sk);
+	dfrag = mptcp_rtx_head(sk);
+	if (!dfrag)
+		goto unlock;
+
+	ssk = mptcp_subflow_get_ref(msk);
+	if (!ssk)
+		goto reset_unlock;
+
+	lock_sock(ssk);
+
+	msg.msg_flags = MSG_DONTWAIT;
+	orig_len = dfrag->data_len;
+	orig_offset = dfrag->offset;
+	orig_write_seq = dfrag->data_seq;
+	while (dfrag->data_len > 0) {
+		ret = mptcp_sendmsg_frag(sk, ssk, &msg, dfrag, &timeo, &mss_now,
+					 &size_goal);
+		if (ret < 0)
+			break;
+
+		copied += ret;
+		dfrag->data_len -= ret;
+	}
+	if (copied)
+		tcp_push(ssk, msg.msg_flags, mss_now, tcp_sk(ssk)->nonagle,
+			 size_goal);
+
+	dfrag->data_seq = orig_write_seq;
+	dfrag->offset = orig_offset;
+	dfrag->data_len = orig_len;
+
+	release_sock(ssk);
+	sock_put(ssk);
+
+reset_unlock:
+	mptcp_set_timeout(sk, ssk);
+	if (!mptcp_timer_pending(sk))
+		mptcp_reset_timer(sk);
+
+unlock:
+	release_sock(sk);
+	sock_put(sk);
+}
+
 static int __mptcp_init_sock(struct sock *sk)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
@@ -805,6 +867,8 @@ static int __mptcp_init_sock(struct sock *sk)
 	INIT_LIST_HEAD(&msk->conn_list);
 	INIT_LIST_HEAD(&msk->rtx_queue);
 
+	INIT_WORK(&msk->rtx_work, mptcp_retransmit);
+
 	/* re-use the csk retrans timer for MPTCP-level retrans */
 	timer_setup(&msk->sk.icsk_retransmit_timer, mptcp_retransmit_timer, 0);
 
@@ -837,6 +901,14 @@ static void __mptcp_clear_xmit(struct sock *sk)
 		dfrag_clear(sk, dfrag);
 }
 
+static void mptcp_cancel_rtx_work(struct sock *sk)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
+	if (cancel_work_sync(&msk->rtx_work))
+		sock_put(sk);
+}
+
 static void mptcp_close(struct sock *sk, long timeout)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
@@ -865,6 +937,8 @@ static void mptcp_close(struct sock *sk, long timeout)
 	__mptcp_clear_xmit(sk);
 	release_sock(sk);
 
+	mptcp_cancel_rtx_work(sk);
+
 	sk_common_release(sk);
 }
 
@@ -873,6 +947,7 @@ static int mptcp_disconnect(struct sock *sk, int flags)
 	lock_sock(sk);
 	__mptcp_clear_xmit(sk);
 	release_sock(sk);
+	mptcp_cancel_rtx_work(sk);
 	return tcp_disconnect(sk, flags);
 }
 
net/mptcp/protocol.h +11 -0
index a14fdc161813..b9d136b01165 100644
@@ -133,6 +133,7 @@ struct mptcp_sock {
 	u32		token;
 	unsigned long	flags;
 	u16		dport;
+	struct work_struct rtx_work;
 	struct list_head conn_list;
 	struct list_head rtx_queue;
 	struct socket	*subflow; /* outgoing connect/listener/!mp_capable */
@@ -158,6 +159,16 @@ static inline struct mptcp_data_frag *mptcp_rtx_tail(const struct sock *sk)
 	return list_last_entry(&msk->rtx_queue, struct mptcp_data_frag, list);
 }
 
+static inline struct mptcp_data_frag *mptcp_rtx_head(const struct sock *sk)
+{
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
+	if (list_empty(&msk->rtx_queue))
+		return NULL;
+
+	return list_first_entry(&msk->rtx_queue, struct mptcp_data_frag, list);
+}
+
 struct subflow_request_sock {
 	struct	tcp_request_sock sk;
 	u8	mp_capable : 1,
Signature · 1 line
2.21.0
#3 lore raw

Re: [MPTCP] [PATCH 2/7] mptcp: update per unacked sequence on pkt reception

In reply to Paolo Abeni
Quoted 35 lines · Paolo Abeni

So that we keep per unacked sequence number consistent; since we update per msk data, use an atomic64 cmpxcgh() to protect against concurrent updates from multiple subflows.

Initialize the snd_una at connect()/accept() time.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
rfc v2 -> v1:
- do msk-level accounting, using atomic64_t

rfc v1 -> rfc v2:
- initialize subflow snd_una on subflow connect()/accept()
---

net/mptcp/options.c  | 39 +++++++++++++++++++++++++++++++++------
net/mptcp/protocol.c |  2 ++
net/mptcp/protocol.h |  1 +
3 files changed, 36 insertions(+), 6 deletions(-)
net/mptcp/options.c +8 -0
index be7afa5aac6a..854dc050c1da 100644
@@ -524,6 +524,26 @@ bool mptcp_synack_options(const struct request_sock *req, unsigned int *size,
 	return false;
 }
 
+static u64 expand_ack(u64 old_ack, u64 cur_ack, bool use_64bit)
+{
+	u32 old_ack32, cur_ack32;
+
+	if (use_64bit)
+		return cur_ack;
+
+	old_ack32 = (u32)old_ack;

net/mptcp/options.c bool mptcp_synack_options(const struct request_sock *req, unsigned int *size, · line 544

Quoted 10 lines
net/mptcp/options.c +10 -0
+	cur_ack32 = (u32)cur_ack;
+	cur_ack = (old_ack & GENMASK_ULL(63, 32)) + cur_ack32;
+	if (unlikely(before(cur_ack32, old_ack32)))
+		return cur_ack + (1LL << 32);
+	return cur_ack;
+}
+
+void update_una(struct mptcp_sock *msk, u64 old_una, u64 new_una)
+{
+	atomic64_cmpxchg(&msk->snd_una, old_una, new_una);

Do we want to silently skip the update if the old_una does not match?

Skipping the update is certainly the right thing if msk->snd_una was changed to be a value after new_una. If msk->snd_una is still behind new_una, then we're ignoring a valid ack.

Back when I was (mistakenly) using cmpxchg with msk->ack_seq, I had this chunk of code:

+ do {
+ if (!before64(old_ack, ack_seq)) {
+ pr_debug("skipping ack update");
+ break;
+ }
+ pr_debug("old_ack=%llu new_ack=%llu", old_ack, ack_seq);
+ } while (!atomic64_try_cmpxchg(&arg->msk->ack_seq, &old_ack,
ack_seq));

Seems like it would be good to use something similar here, maybe limiting the retries.

Quoted 6 lines
net/mptcp/options.c +2 -0
+}
+
 void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 			    struct tcp_options_received *opt_rx)
 {

net/mptcp/options.c void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb, · line 574

Quoted 8 lines
net/mptcp/options.c +5 -0
@@ -547,6 +567,19 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 	if (!mp_opt->dss)
 		return;
 
+	/* we can't wait for recvmsg() to update the ack_seq, otherwise
+	 * monodirectional flows will stuck
+	 */
+	if (msk && mp_opt->use_ack) {
+		u64 snd_una = atomic64_read(&msk->snd_una);

After our discussion of memory barriers in the "Make MPTCP socket block/wakeup ignore sk_receive_queue" patch, I wanted to double-check this use. Documentation/atomic_t.txt says that "non-RMW" operations like this read are unordered, and need smp_mb__before_atomic() or smp_mb__after_atomic(). It seems like we need to worry about ordering because we have a locked subflow here and an msk lock is not held by this thread, and there may be other subflows trying to update snd_una on other CPUs.

Quoted 29 lines
net/mptcp/options.c +8 -6
+		u64 new_snd_una;
+
+		new_snd_una = expand_ack(snd_una, mp_opt->data_ack,
+					 mp_opt->ack64);
+		if (after64(new_snd_una, snd_una))
+			update_una(msk, snd_una, new_snd_una);
+	}
+
 	mpext = skb_ext_add(skb, SKB_EXT_MPTCP);
 	if (!mpext)
 		return;
@@ -563,12 +596,6 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 		mpext->use_checksum = mp_opt->use_checksum;
 	}
 
-	if (mp_opt->use_ack) {
-		mpext->data_ack = mp_opt->data_ack;
-		mpext->use_ack = 1;
-		mpext->ack64 = mp_opt->ack64;
-	}
-
 	mpext->data_fin = mp_opt->data_fin;
 
 	if (msk)
net/mptcp/protocol.c +0 -0
index 15bd54f43d4b..52e39fbb217d 100644

net/mptcp/protocol.c static struct sock *mptcp_accept(struct sock *sk, int flags, int *err, · line 699

Quoted 4 lines
net/mptcp/protocol.c +1 -0
@@ -696,6 +696,7 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,
 
 		crypto_key_sha1(msk->remote_key, NULL, &ack_seq);
 		msk->write_seq = subflow->idsn + 1;
+		atomic64_set(&msk->snd_una, msk->write_seq);

Does the msk lock give us the ordering we need here?

Thanks,
Mat

Quoted 31 lines
net/mptcp/protocol.c +1 -0
 		ack_seq++;
 		msk->ack_seq = ack_seq;
 		subflow->map_seq = ack_seq;
@@ -833,6 +834,7 @@ void mptcp_finish_connect(struct sock *sk, int mp_capable)
 
 		crypto_key_sha1(msk->remote_key, NULL, &ack_seq);
 		msk->write_seq = subflow->idsn + 1;
+		atomic64_set(&msk->snd_una, msk->write_seq);
 		ack_seq++;
 		msk->ack_seq = ack_seq;
 		subflow->map_seq = ack_seq;
net/mptcp/protocol.h +1 -0
index 8c36a379f67d..e7c29007f0fe 100644
@@ -119,6 +119,7 @@ struct mptcp_sock {
 	u64		remote_key;
 	u64		write_seq;
 	u64		ack_seq;
+	atomic64_t	snd_una;
 	u32		token;
 	unsigned long	flags;
 	u16		dport;
Signature · 6 lines
2.21.0

_______________________________________________
mptcp mailing list
mptcp(a)lists.01.org
https://lists.01.org/mailman/listinfo/mptcp

--
Mat Martineau
Intel

author
#4 lore raw

Re: [MPTCP] [PATCH 2/7] mptcp: update per unacked sequence on pkt reception

Quoted 22 lines · Mat Martineau

net/mptcp/options.c bool mptcp_synack_options(const struct request_sock *req, unsigned int *size, · line 544

Quoted 3 lines
net/mptcp/options.c +3 -0
+void update_una(struct mptcp_sock *msk, u64 old_una, u64 new_una)
+{
+	atomic64_cmpxchg(&msk->snd_una, old_una, new_una);

Do we want to silently skip the update if the old_una does not match?

Skipping the update is certainly the right thing if msk->snd_una was changed to be a value after new_una. If msk->snd_una is still behind new_una, then we're ignoring a valid ack.

Back when I was (mistakenly) using cmpxchg with msk->ack_seq, I had this chunk of code:

(quoted diff) +7 -0
+       do {
+               if (!before64(old_ack, ack_seq)) {
+                       pr_debug("skipping ack update");
+                       break;
+               }
+               pr_debug("old_ack=%llu new_ack=%llu", old_ack, ack_seq);
+       } while (!atomic64_try_cmpxchg(&arg->msk->ack_seq, &old_ack, 

ack_seq));

Quoted 2 lines

Seems like it would be good to use something similar here, maybe limiting the retries.

Yep, I thought a bit about introdicing something similar, but for this submission I preferred keep the code as simple as possible. At worse we will need a retransmission or reinjection (but the occurence looks quite unlikely). I can add the loop in the next iteration. I don't think bounding it is required, even a malicius peer can push packets b2b close enough to let us spinning here.

Quoted 9 lines
Quoted 9 lines
(quoted diff) +2 -0
+}
+

void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
struct tcp_options_received *opt_rx)
{

net/mptcp/options.c +0 -0
@@ -547,6 +567,19 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 	if (!mp_opt->dss)
 		return;
 
Quoted 14 lines

net/mptcp/options.c void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb, · line 574

Quoted 5 lines
net/mptcp/options.c +5 -0
+	/* we can't wait for recvmsg() to update the ack_seq, otherwise
+	 * monodirectional flows will stuck
+	 */
+	if (msk && mp_opt->use_ack) {
+		u64 snd_una = atomic64_read(&msk->snd_una);

After our discussion of memory barriers in the "Make MPTCP socket block/wakeup ignore sk_receive_queue" patch, I wanted to double-check this use. Documentation/atomic_t.txt says that "non-RMW" operations like this read are unordered, and need smp_mb__before_atomic() or smp_mb__after_atomic(). It seems like we need to worry about ordering because we have a locked subflow here and an msk lock is not held by this thread, and there may be other subflows trying to update snd_una on other CPUs.

I *think* atomic64_read() is enough here, mainly because I don't see other read operations we need to enforce an order? What I'm missing???

Quoted 11 lines

net/mptcp/protocol.c static struct sock *mptcp_accept(struct sock *sk, int flags, int *err, · line 699

Quoted 9 lines
net/mptcp/protocol.c +1 -0
index 15bd54f43d4b..52e39fbb217d 100644
@@ -696,6 +696,7 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,
 
 		crypto_key_sha1(msk->remote_key, NULL, &ack_seq);
 		msk->write_seq = subflow->idsn + 1;
+		atomic64_set(&msk->snd_una, msk->write_seq);

Does the msk lock give us the ordering we need here?

Same here, in respect to which other write operations we want to sync here?

Thanks,

Paolo

#5 lore raw

Re: [MPTCP] [PATCH 2/7] mptcp: update per unacked sequence on pkt reception

Quoted 26 lines · Paolo Abeni
Quoted 22 lines · Mat Martineau

net/mptcp/options.c bool mptcp_synack_options(const struct request_sock *req, unsigned int *size, · line 544

Quoted 3 lines
net/mptcp/options.c +3 -0
+void update_una(struct mptcp_sock *msk, u64 old_una, u64 new_una)
+{
+	atomic64_cmpxchg(&msk->snd_una, old_una, new_una);

Do we want to silently skip the update if the old_una does not match?

Skipping the update is certainly the right thing if msk->snd_una was changed to be a value after new_una. If msk->snd_una is still behind new_una, then we're ignoring a valid ack.

Back when I was (mistakenly) using cmpxchg with msk->ack_seq, I had this chunk of code:

(quoted diff) +7 -0
+       do {
+               if (!before64(old_ack, ack_seq)) {
+                       pr_debug("skipping ack update");
+                       break;
+               }
+               pr_debug("old_ack=%llu new_ack=%llu", old_ack, ack_seq);
+       } while (!atomic64_try_cmpxchg(&arg->msk->ack_seq, &old_ack,

ack_seq));

Quoted 2 lines

Seems like it would be good to use something similar here, maybe limiting the retries.

Quoted 7 lines

Yep, I thought a bit about introdicing something similar, but for this submission I preferred keep the code as simple as possible. At worse we will need a retransmission or reinjection (but the occurence looks quite unlikely). I can add the loop in the next iteration. I don't think bounding it is required, even a malicius peer can push packets b2b close enough to let us spinning here.

Ok, I'm fine with adding the loop in the next iteration.

Quoted 24 lines
Quoted 9 lines
Quoted 9 lines
(quoted diff) +2 -0
+}
+

void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
struct tcp_options_received *opt_rx)
{

net/mptcp/options.c +0 -0
@@ -547,6 +567,19 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 	if (!mp_opt->dss)
 		return;
 
Quoted 14 lines

net/mptcp/options.c void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb, · line 574

Quoted 5 lines
net/mptcp/options.c +5 -0
+	/* we can't wait for recvmsg() to update the ack_seq, otherwise
+	 * monodirectional flows will stuck
+	 */
+	if (msk && mp_opt->use_ack) {
+		u64 snd_una = atomic64_read(&msk->snd_una);

After our discussion of memory barriers in the "Make MPTCP socket block/wakeup ignore sk_receive_queue" patch, I wanted to double-check this use. Documentation/atomic_t.txt says that "non-RMW" operations like this read are unordered, and need smp_mb__before_atomic() or smp_mb__after_atomic(). It seems like we need to worry about ordering because we have a locked subflow here and an msk lock is not held by this thread, and there may be other subflows trying to update snd_una on other CPUs.

Quoted 3 lines

I *think* atomic64_read() is enough here, mainly because I don't see other read operations we need to enforce an order? What I'm missing???

I don't think you're missing anything, the misunderstanding was on my part.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
net/mptcp/protocol.c +0 -0
index 15bd54f43d4b..52e39fbb217d 100644
@@ -696,6 +696,7 @@ static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,
 
Quoted 8 lines
Quoted 5 lines

net/mptcp/protocol.c static struct sock *mptcp_accept(struct sock *sk, int flags, int *err, · line 699

Quoted 3 lines
net/mptcp/protocol.c +1 -0
 		crypto_key_sha1(msk->remote_key, NULL, &ack_seq);
 		msk->write_seq = subflow->idsn + 1;
+		atomic64_set(&msk->snd_una, msk->write_seq);

Does the msk lock give us the ordering we need here?

Same here, in respect to which other write operations we want to sync here?

I was looking at any of the changes to msk->snd_una that happen without the msk lock held. Looking at it again, I'm pretty sure this is fine - all of these values need to be set before any data is sent or received, and both the subflow and mptcp sockets are locked. Sorry for the confusion.

--
Mat Martineau
Intel