Skip to contentSkip to conversation

mptcp: add and use wmem_queued accounting

PATCH New Florian Westphal 1 participant 5 messages Nov 12, 2019

Subsystems (2)
lore
Share & tools
lore

Florian Westphal

author
#0 lore raw

v2 cover mptcp: add and use wmem_queued accounting

This is v2 of the wmem accouting patch set.

I've dropped the ack_seq patches, we can resurrect them later on if needed.

After adding wmem thresholds, mptcp can deadlock at connection level when both sides call write() at the same time and sndbuf is exhausted.

This won't happen in the selftests because it only sends small amounts of data at a time and will always call read() if data is available. It could happen with code that sends large amount of data at once though, or when sndbuf is reduced via setsockopt:

Unlike TCP, which acknowledges in-sequence bytes queued to the socket, MPTCP updates its ack_seq only when recv() is called.

In case both peers are blocked in write, no updates of the mptcp ack sequence space occurs. This in turn can prevent both peers from making any progress: even if all data is queued on the peer, it will look like no data arrived and thus no wmem can be freed up for the write() to complete.

Resolving this will need more work to get pushed into the data_ready callback, which is hard because of need to synchronize with other subflows used by same mptcp socket.

The following changes since commit 6e61dd3c078fe69ac2e442a9251d771a61dcc62f:

subflow: wake parent mptcp socket on subflow state change (2019-11-08 15:22:58 +0000)

are available in the Git repository at:

git://git.breakpoint.cc/fw/mptcp-next.git wmem_acct_06

for you to fetch changes up to eea081f8828bbe2c503873319374a7666c6b6af2:

sendmsg: block until mptcp sk is writeable (2019-11-12 14:53:29 +0100)

----------------------------------------------------------------
Florian Westphal (4):
mptcp: add wmem_queued accounting
mptcp: allow partial cleaning of rtx head dfrag
mptcp: add and use mptcp RTX flag
sendmsg: block until mptcp sk is writeable

 net/mptcp/options.c  |  2 +-
 net/mptcp/protocol.c | 77 +++++++++++++++++++++++++++++++++++++++++++++-------
 net/mptcp/protocol.h |  3 +-
 3 files changed, 70 insertions(+), 12 deletions(-)
author
#1 lore raw

PATCH v2 1/4 mptcp: add wmem_queued accounting

Peer could ack data at TCP level but refrain from sending mptcp-level ACKs. This could result in a growing the mptcp socket backlog indefinitely.

We should thus block mptcp_sendmsg until the peer has acked some of the sent data.

In order to be able to do so, increment the mptcp socket wmem_queued counter on memory allocation and decrement it when releasing the memory on mptcp-level ack reception.

Because TCP performns sndbuf auto-tuning up to tcp_wmem_max[2], make this the mptcp sk_sndbuf limit.

In the future we could add experiment with autotuning as TCP does in tcp_sndbuf_expand().

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +8 -1
index 7bea6b62f66e..65f10bb372aa 100644
@@ -139,8 +139,11 @@ static inline bool mptcp_frag_can_collapse_to(const struct mptcp_sock *msk,
 
 static void dfrag_clear(struct sock *sk, struct mptcp_data_frag *dfrag)
 {
+	int len = dfrag->data_len + dfrag->overhead;
+
 	list_del(&dfrag->list);
-	sk_mem_uncharge(sk, dfrag->data_len + dfrag->overhead);
+	sk_mem_uncharge(sk, len);
+	sk_wmem_queued_add(sk, -len);
 	put_page(dfrag->page);
 }
 
@@ -304,6 +307,9 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 		if (!dfrag_collapsed) {
 			get_page(dfrag->page);
 			list_add_tail(&dfrag->list, &msk->rtx_queue);
+			sk_wmem_queued_add(sk, frag_truesize);
+		} else {
+			sk_wmem_queued_add(sk, ret);
 		}
 
 		/* charge data on mptcp rtx queue to the master socket
@@ -711,6 +717,7 @@ static int mptcp_init_sock(struct sock *sk)
 		return ret;
 
 	sk_sockets_allocated_inc(sk);
+	sk->sk_sndbuf = sock_net(sk)->ipv4.sysctl_tcp_wmem[2];
 
 	return 0;
 }
Signature · 1 line
2.23.0
author
#2 lore raw

PATCH v2 2/4 mptcp: allow partial cleaning of rtx head dfrag

In reply to Florian Westphal

After adding wmem accouting for the mptcp socket we could get into a situation where the mptcp socket can't transmit more data, and mptcp_clean_una doesn't reduce wmem even if snd_una has advanced because it currently will only remove entire dfrags.

Allow advancing the dfrag head sequence and reduce wmem, even though this isn't correct (as we can't release the page).

Because we will soon block on mptcp sk in case wmem is too large, call sk_stream_write_space() in case we reduced the backlog so userspace task blocked in sendmsg or poll will be woken up.

This isn't an issue if the send buffer is large, but it is when SO_SNDBUF is used to reduce it to a lower value.

Note we can still get a deadlock for low SO_SNDBUF values in case both sides of the connection write to the socket: both could be blocked due to wmem being too small -- and current mptcp stack will only increment mptcp ack_seq on recv.

This doesn't happen with the selftest as it uses poll() and will always call recv if there is data to read.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +25 -3
index 65f10bb372aa..b93050056195 100644
@@ -137,13 +137,18 @@ static inline bool mptcp_frag_can_collapse_to(const struct mptcp_sock *msk,
 		df->data_seq + df->data_len == msk->write_seq;
 }
 
+static void dfrag_uncharge(struct sock *sk, int len)
+{
+	sk_mem_uncharge(sk, len);
+	sk_wmem_queued_add(sk, -len);
+}
+
 static void dfrag_clear(struct sock *sk, struct mptcp_data_frag *dfrag)
 {
 	int len = dfrag->data_len + dfrag->overhead;
 
 	list_del(&dfrag->list);
-	sk_mem_uncharge(sk, len);
-	sk_wmem_queued_add(sk, -len);
+	dfrag_uncharge(sk, len);
 	put_page(dfrag->page);
 }
 
@@ -152,14 +157,31 @@ static void mptcp_clean_una(struct sock *sk)
 	struct mptcp_sock *msk = mptcp_sk(sk);
 	struct mptcp_data_frag *dtmp, *dfrag;
 	u64 snd_una = atomic64_read(&msk->snd_una);
+	bool cleaned = false;
 
 	list_for_each_entry_safe(dfrag, dtmp, &msk->rtx_queue, list) {
 		if (after64(dfrag->data_seq + dfrag->data_len, snd_una))
 			break;
 
 		dfrag_clear(sk, dfrag);
+		cleaned = true;
+	}
+
+	dfrag = mptcp_rtx_head(sk);
+	if (dfrag && after64(snd_una, dfrag->data_seq)) {
+		u64 delta = dfrag->data_seq + dfrag->data_len - snd_una;
+
+		dfrag->data_seq += delta;
+		dfrag->data_len -= delta;
+
+		dfrag_uncharge(sk, delta);
+		cleaned = true;
+	}
+
+	if (cleaned) {
+		sk_mem_reclaim_partial(sk);
+		sk_stream_write_space(sk);
 	}
-	sk_mem_reclaim_partial(sk);
 }
 
 /* ensure we get enough memory for the frag hdr, beyond some minimal amount of
Signature · 1 line
2.23.0
author
#3 lore raw

PATCH v2 3/4 mptcp: add and use mptcp RTX flag

In reply to Florian Westphal

This is needed in the (unlikely) case that userspace is blocked in mptcp_sendmsg because wmem is exhausted.

In that case, only the rtx work queue will clean the rtx backlog, but it could take several milliseconds until it runs next. So, allow it to get scheduled as soon as possible so wmem can be reclaimed if the mptcp socket sndbuf is exhausted.

Because such quick-schedule should not cause retransmits, add a flag that indicates when the work queue has been scheduled on behalf of the retransmit timer.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/options.c  |  2 +-
 net/mptcp/protocol.c | 26 ++++++++++++++++++++------
 net/mptcp/protocol.h |  3 ++-
 3 files changed, 23 insertions(+), 8 deletions(-)
net/mptcp/options.c +1 -1
index 80dbe7662cea..b64fb5dbc2a1 100644
@@ -583,7 +583,7 @@ static void update_una(struct mptcp_sock *msk,
 		old_snd_una = atomic64_cmpxchg(&msk->snd_una, snd_una,
 					       new_snd_una);
 		if (old_snd_una == snd_una) {
-			mptcp_reset_timer((struct sock *)msk);
+			mptcp_data_acked((struct sock *)msk);
 			break;
 		}
 	}
net/mptcp/protocol.c +20 -6
index b93050056195..68171a539a02 100644
@@ -40,7 +40,7 @@ static bool mptcp_timer_pending(struct sock *sk)
 	return timer_pending(&inet_csk(sk)->icsk_retransmit_timer);
 }
 
-void mptcp_reset_timer(struct sock *sk)
+static void mptcp_reset_timer(struct sock *sk)
 {
 	struct inet_connection_sock *icsk = inet_csk(sk);
 	unsigned long tout;
@@ -52,6 +52,15 @@ void mptcp_reset_timer(struct sock *sk)
 	sk_reset_timer(sk, &icsk->icsk_retransmit_timer, jiffies + tout);
 }
 
+void mptcp_data_acked(struct sock *sk)
+{
+	mptcp_reset_timer(sk);
+
+	if (!sk_stream_is_writeable(sk) &&
+	    schedule_work(&mptcp_sk(sk)->rtx_work))
+		sock_hold(sk);
+}
+
 static void mptcp_stop_timer(struct sock *sk)
 {
 	struct inet_connection_sock *icsk = inet_csk(sk);
@@ -623,6 +632,7 @@ static void mptcp_retransmit_handler(struct sock *sk)
 	if (atomic64_read(&msk->snd_una) == msk->write_seq) {
 		mptcp_stop_timer(sk);
 	} else {
+		set_bit(MPTCP_WORK_RTX, &msk->flags);
 		if (schedule_work(&msk->rtx_work))
 			sock_hold(sk);
 	}
@@ -647,7 +657,7 @@ static void mptcp_retransmit_timer(struct timer_list *t)
 	sock_put(sk);
 }
 
-static void mptcp_retransmit(struct work_struct *work)
+static void mptcp_worker(struct work_struct *work)
 {
 	int orig_len, orig_offset, ret, mss_now = 0, size_goal = 0;
 	struct mptcp_data_frag *dfrag;
@@ -663,6 +673,10 @@ static void mptcp_retransmit(struct work_struct *work)
 
 	lock_sock(sk);
 	mptcp_clean_una(sk);
+
+	if (!test_and_clear_bit(MPTCP_WORK_RTX, &msk->flags))
+		goto unlock;
+
 	dfrag = mptcp_rtx_head(sk);
 	if (!dfrag)
 		goto unlock;
@@ -715,7 +729,7 @@ static int __mptcp_init_sock(struct sock *sk)
 	INIT_LIST_HEAD(&msk->conn_list);
 	INIT_LIST_HEAD(&msk->rtx_queue);
 
-	INIT_WORK(&msk->rtx_work, mptcp_retransmit);
+	INIT_WORK(&msk->rtx_work, mptcp_worker);
 
 	/* re-use the csk retrans timer for MPTCP-level retrans */
 	timer_setup(&msk->sk.icsk_retransmit_timer, mptcp_retransmit_timer, 0);
@@ -755,7 +769,7 @@ static void __mptcp_clear_xmit(struct sock *sk)
 		dfrag_clear(sk, dfrag);
 }
 
-static void mptcp_cancel_rtx_work(struct sock *sk)
+static void mptcp_cancel_work(struct sock *sk)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
 
@@ -792,7 +806,7 @@ static void mptcp_close(struct sock *sk, long timeout)
 	__mptcp_clear_xmit(sk);
 	release_sock(sk);
 
-	mptcp_cancel_rtx_work(sk);
+	mptcp_cancel_work(sk);
 
 	sk_common_release(sk);
 }
@@ -802,7 +816,7 @@ static int mptcp_disconnect(struct sock *sk, int flags)
 	lock_sock(sk);
 	__mptcp_clear_xmit(sk);
 	release_sock(sk);
-	mptcp_cancel_rtx_work(sk);
+	mptcp_cancel_work(sk);
 	return tcp_disconnect(sk, flags);
 }
 
net/mptcp/protocol.h +2 -1
index 83b06382e56a..bb599cd3a738 100644
@@ -76,6 +76,7 @@
 
 /* MPTCP socket flags */
 #define MPTCP_DATA_READY	BIT(0)
+#define MPTCP_WORK_RTX		BIT(1)
 
 static inline __be32 mptcp_option(u8 subopt, u8 len, u8 nib, u8 field)
 {
@@ -290,7 +291,7 @@ void mptcp_get_options(const struct sk_buff *skb,
 
 void mptcp_finish_connect(struct sock *sk, int mp_capable);
 void mptcp_finish_join(struct sock *sk);
-void mptcp_reset_timer(struct sock *sk);
+void mptcp_data_acked(struct sock *sk);
 
 int mptcp_token_new_request(struct request_sock *req);
 void mptcp_token_destroy_request(u32 token);
Signature · 1 line
2.23.0
author
#4 lore raw

PATCH v2 4/4 sendmsg: block until mptcp sk is writeable

In reply to Florian Westphal

This disables transmit of new data until the peer has acked enough mptcp data to get below the wspace write threshold (more than half of wspace upperlimit is available again).

Also have poll not report EPOLLOUT in this case, its not relevant if a subflow is writeable.

The latter is a temporary workaround that is needed because mptcp_poll walks the subflows and calls __tcp_poll on each of them. Because subflow ssk is usually writable, we will have to undo-that if the mptcp sndbuf is exhausted. This won't be needed anymore once __tcp_poll is removed, I am working on this.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
net/mptcp/protocol.c +16 -2
index 68171a539a02..4da4c702b1a5 100644
@@ -406,6 +406,18 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		return ret;
 	}
 
+	timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
+
+	mptcp_clean_una(sk);
+
+	while (!sk_stream_is_writeable(sk)) {
+		ret = sk_stream_wait_memory(sk, &timeo);
+		if (ret)
+			goto out;
+
+		mptcp_clean_una(sk);
+	}
+
 	ssk = mptcp_subflow_get(msk);
 	if (!ssk) {
 		release_sock(sk);
@@ -421,8 +433,6 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	pr_debug("conn_list->subflow=%p", ssk);
 
 	lock_sock(ssk);
-	mptcp_clean_una(sk);
-	timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
 	while (msg_data_left(msg)) {
 		ret = mptcp_sendmsg_frag(sk, ssk, msg, NULL, &timeo, &mss_now,
 					 &size_goal);
@@ -1315,6 +1325,10 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,
 		tcp_sock = mptcp_subflow_tcp_socket(subflow);
 		ret |= __tcp_poll(tcp_sock->sk);
 	}
+
+	if (!sk_stream_is_writeable(sk))
+		ret &= ~(EPOLLOUT|EPOLLWRNORM);
+
 	release_sock(sk);
 
 	return ret;
Signature · 1 line
2.23.0