Skip to contentSkip to conversation

mptcp: don't sleep on ssk with msk lock held

PATCH In review Florian Westphal 3 participants 15 messages May 6, 2020 – May 8, 2020

Subsystems (2)
lore
Share & tools
lore

Florian Westphal

author
#0 lore raw

cover mptcp: don't sleep on ssk with msk lock held

This series reworks mptcp_sendmsg logic to avoid blocking on the subflow socket in an attempt to address/resolve:
https://github.com/multipath-tcp/mptcp_net-next/issues/7

It does so by removing the wait loop from mptcp_sendmsg_frag helper.

In order to do that, it moves prerequisites into the callers.

The worker can just reschedule in case no subflow socket is ready.

The sendmsg path can use the existing wait logic.

Because large send requests can result in multiple mptcp_sendmsg_frag calls from mptcp_sendmsg, we may need to restart the socket lookup in case subflow can't accept more data or memory is low.

Doing so blocks on the mptcp socket, and existing wait handling releases the msk lock while blocking.

Lastly, no need to use ATOMIC allocations for extension allocation, so allow to pass gfp_t arg to the helper instead of hard-coded ATOMIC in the core.

Comments welcome.

Florian Westphal (7):
mptcp: avoid blocking in tcp_sendpages
mptcp: make sure timer is inited/started once data is pending for transmission
mptcp: avoid blocking in tcp_sendpages due to skb alloc
mptcp: fill skb extension cache outside of mptcp_sendmsg_frag
mptcp: fill skb page frag cache outside of mptcp_sendmsg_frag
mptcp: remove inner wait loop from mptcp_sendmsg_frag
net: allow __skb_ext_alloc to sleep

 include/linux/skbuff.h |   2 +-
 net/core/skbuff.c      |   8 ++--
 net/mptcp/protocol.c   | 100 ++++++++++++++++++++++++++++++++++++-------------
 3 files changed, 81 insertions(+), 29 deletions(-)
author
#1 lore raw

PATCH 1/7 mptcp: avoid blocking in tcp_sendpages

The transmit loop continues to xmit new data until an error is returned or all data was transmitted.

For the blocking i/o case, this means that tcp_sendpages() may block on the subflow until more space becomes available, i.e. we end up sleeping with the mptcp socket lock held.

Instead we should check if a different subflow is ready to be used.

This restarts the subflow sk lookup when the tx operation succeeded and the tcp subflow can't accept more data or if tcp_sendpages indicates -EAGAIN on a blocking mptcp socket.

In case all subflows are busy, the existing logic will wait until a subflow is ready, releasing the mptcp level socket while doing so.

The mptcp worker already sets DONTWAIT, so no need to make changes there.

Reported-by Paolo Abeni <pabeni(a)redhat.com>
Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +20 -3
index b22a63ba2348..25427797753b 100644
@@ -590,7 +590,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	 * access the skb after the sendpages call
 	 */
 	ret = do_tcp_sendpages(ssk, page, offset, psize,
-			       msg->msg_flags | MSG_SENDPAGE_NOTLAST);
+			       msg->msg_flags | MSG_SENDPAGE_NOTLAST | MSG_DONTWAIT);
 	if (ret <= 0)
 		return ret;
 
@@ -714,6 +714,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	struct socket *ssock;
 	size_t copied = 0;
 	struct sock *ssk;
+	bool tx_ok;
 	long timeo;
 
 	if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL))
@@ -738,6 +739,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		return ret >= 0 ? ret + copied : (copied ? copied : ret);
 	}
 
+restart:
 	mptcp_clean_una(sk);
 
 	__mptcp_flush_join_list(msk);
@@ -759,11 +761,17 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	pr_debug("conn_list->subflow=%p", ssk);
 
 	lock_sock(ssk);
-	while (msg_data_left(msg)) {
+	tx_ok = msg_data_left(msg);
+	while (tx_ok) {
 		ret = mptcp_sendmsg_frag(sk, ssk, msg, NULL, &timeo, &mss_now,
 					 &size_goal);
-		if (ret < 0)
+		if (ret < 0) {
+			if (ret == -EAGAIN && timeo > 0) {
+				release_sock(ssk);
+				goto restart;
+			}
 			break;
+		}
 		if (ret == 0 && unlikely(__mptcp_needs_tcp_fallback(msk))) {
 			/* Can happen for passive sockets:
 			 * 3WHS negotiated MPTCP, but first packet after is
@@ -777,6 +785,15 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		}
 
 		copied += ret;
+		tx_ok = msg_data_left(msg);
+		if (!tx_ok)
+			break;
+		if (!sk_stream_memory_free(ssk)) {
+			tcp_push(ssk, msg->msg_flags, mss_now,
+				 tcp_sk(ssk)->nonagle, size_goal);
+			release_sock(ssk);
+			goto restart;
+		}
 	}
 
 	mptcp_set_timeout(sk, ssk);
Signature · 1 line
2.26.2
#2 lore raw

[MPTCP] Re: [PATCH mptcp-next 1/7] mptcp: avoid blocking in tcp_sendpages

Quoted 9 lines · Florian Westphal

The transmit loop continues to xmit new data until an error is returned or all data was transmitted.

For the blocking i/o case, this means that tcp_sendpages() may block on the subflow until more space becomes available, i.e. we end up sleeping with the mptcp socket lock held.

Instead we should check if a different subflow is ready to be used.

Keep in mind that configurable packet schedulers (when eventually implemented) may want to make other decisions when one subflow gets blocked, even if other subflows have available space. One specific subflow might be favored due to better latency, or a redundant scheduler could send the same data on all the available subflows.

I think this patch set makes it easier to make mptcp_subflow_get_send configurable and to have the scheduler reselect when a subflow is out of space, so that's good!

Mat

Quoted 87 lines

This restarts the subflow sk lookup when the tx operation succeeded and the tcp subflow can't accept more data or if tcp_sendpages indicates -EAGAIN on a blocking mptcp socket.

In case all subflows are busy, the existing logic will wait until a subflow is ready, releasing the mptcp level socket while doing so.

The mptcp worker already sets DONTWAIT, so no need to make changes there.

Reported-by Paolo Abeni <pabeni(a)redhat.com>
Signed-off-by Florian Westphal <fw(a)strlen.de>

---
net/mptcp/protocol.c | 23 ++++++++++++++++++++---

1 file changed, 20 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +20 -3
index b22a63ba2348..25427797753b 100644
@@ -590,7 +590,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	 * access the skb after the sendpages call
 	 */
 	ret = do_tcp_sendpages(ssk, page, offset, psize,
-			       msg->msg_flags | MSG_SENDPAGE_NOTLAST);
+			       msg->msg_flags | MSG_SENDPAGE_NOTLAST | MSG_DONTWAIT);
 	if (ret <= 0)
 		return ret;
 
@@ -714,6 +714,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	struct socket *ssock;
 	size_t copied = 0;
 	struct sock *ssk;
+	bool tx_ok;
 	long timeo;
 
 	if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL))
@@ -738,6 +739,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		return ret >= 0 ? ret + copied : (copied ? copied : ret);
 	}
 
+restart:
 	mptcp_clean_una(sk);
 
 	__mptcp_flush_join_list(msk);
@@ -759,11 +761,17 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	pr_debug("conn_list->subflow=%p", ssk);
 
 	lock_sock(ssk);
-	while (msg_data_left(msg)) {
+	tx_ok = msg_data_left(msg);
+	while (tx_ok) {
 		ret = mptcp_sendmsg_frag(sk, ssk, msg, NULL, &timeo, &mss_now,
 					 &size_goal);
-		if (ret < 0)
+		if (ret < 0) {
+			if (ret == -EAGAIN && timeo > 0) {
+				release_sock(ssk);
+				goto restart;
+			}
 			break;
+		}
 		if (ret == 0 && unlikely(__mptcp_needs_tcp_fallback(msk))) {
 			/* Can happen for passive sockets:
 			 * 3WHS negotiated MPTCP, but first packet after is
@@ -777,6 +785,15 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		}
 
 		copied += ret;
+		tx_ok = msg_data_left(msg);
+		if (!tx_ok)
+			break;
+		if (!sk_stream_memory_free(ssk)) {
+			tcp_push(ssk, msg->msg_flags, mss_now,
+				 tcp_sk(ssk)->nonagle, size_goal);
+			release_sock(ssk);
+			goto restart;
+		}
 	}
 
 	mptcp_set_timeout(sk, ssk);
Signature · 4 lines
2.26.2
_______________________________________________
mptcp mailing list -- mptcp(a)lists.01.org
To unsubscribe send an email to mptcp-leave(a)lists.01.org

--
Mat Martineau
Intel

author
#3 lore raw

PATCH 2/7 mptcp: make sure timer is inited/started once data is pending for transmission

In reply to Florian Westphal

The mptcp_connect self-tests with additional '-m mmap' argument triggers a (harmless) WARN_ON_ONCE splat in mptcp_reset_timer() as the mptcp timer is not initialized yet.

This is because the timer is only set up after the transmit loop, but consecutive iterations can trigger the splat.

Make sure the timer is set as soon as we pass the first chunk of data.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
net/mptcp/protocol.c +5 -2
index 25427797753b..ab6599dfda59 100644
@@ -785,6 +785,11 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		}
 
 		copied += ret;
+		mptcp_set_timeout(sk, ssk);
+		/* start the timer, if it's not pending */
+		if (!mptcp_timer_pending(sk))
+			mptcp_reset_timer(sk);
+
 		tx_ok = msg_data_left(msg);
 		if (!tx_ok)
 			break;
@@ -796,12 +801,10 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		}
 	}
 
-	mptcp_set_timeout(sk, ssk);
 	if (copied) {
 		ret = copied;
 		tcp_push(ssk, msg->msg_flags, mss_now, tcp_sk(ssk)->nonagle,
 			 size_goal);
-
 		/* start the timer, if it's not pending */
 		if (!mptcp_timer_pending(sk))
 			mptcp_reset_timer(sk);
Signature · 1 line
2.26.2
author
#4 lore raw

PATCH 3/7 mptcp: avoid blocking in tcp_sendpages due to skb alloc

In reply to Florian Westphal

If memory is tight, tcp_sendpages may fail because it can't allocate the skb head.

Now that we pass DONTWAIT to tcp_sendpages it won't block anymore on the subflow socket when this happens.

We can avoid such allocation failure in tcp code by preallocating an skb skb when we pick the subflow to use.

If it fails, we can wait on the mptcp socket until memory becomes available.

Suggested-by Paolo Abeni <pabeni(a)redhat.com>
Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +28 -1
index ab6599dfda59..f28c131c010e 100644
@@ -488,6 +488,22 @@ mptcp_carve_data_frag(const struct mptcp_sock *msk, struct page_frag *pfrag,
 	return dfrag;
 }
 
+static bool mptcp_sendmsg_alloc_skb(struct sock *sk)
+{
+	struct sk_buff *skb;
+
+	if (sk->sk_tx_skb_cache)
+		return true;
+
+	skb = alloc_skb_fclone(sk->sk_prot->max_header, sk->sk_allocation);
+	if (skb) {
+		skb_reserve(skb, sk->sk_prot->max_header);
+		sk->sk_tx_skb_cache = skb;
+	}
+
+	return sk->sk_tx_skb_cache != NULL;
+}
+
 static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 			      struct msghdr *msg, struct mptcp_data_frag *dfrag,
 			      long *timeo, int *pmss_now,
@@ -504,6 +520,11 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	u64 *write_seq;
 	size_t psize;
 
+	if (!ssk->sk_tx_skb_cache) {
+		ssk->sk_tx_skb_cache = sk->sk_tx_skb_cache;
+		sk->sk_tx_skb_cache = NULL;
+	}
+
 	/* use the mptcp page cache so that we can easily move the data
 	 * from one substream to another, but do per subflow memory accounting
 	 * Note: pfrag is used only !retransmission, but the compiler if
@@ -656,9 +677,13 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 {
 	struct mptcp_subflow_context *subflow;
+	struct sock *sk = (struct sock *)msk;
 	struct sock *backup = NULL;
 
-	sock_owned_by_me((const struct sock *)msk);
+	sock_owned_by_me(sk);
+
+	if (!mptcp_sendmsg_alloc_skb(sk))
+		return NULL;
 
 	mptcp_for_each_subflow(msk, subflow) {
 		struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
@@ -1478,6 +1503,7 @@ static void mptcp_destroy(struct sock *sk)
 	if (msk->cached_ext)
 		__skb_ext_put(msk->cached_ext);
 
+	kfree_skb(sk->sk_tx_skb_cache);
 	sk_sockets_allocated_dec(sk);
 }
 
@@ -1688,6 +1714,7 @@ static struct proto mptcp_prot = {
 	.hash		= inet_hash,
 	.unhash		= inet_unhash,
 	.get_port	= mptcp_get_port,
+	.max_header	= MAX_TCP_HEADER,
 	.sockets_allocated	= &mptcp_sockets_allocated,
 	.memory_allocated	= &tcp_memory_allocated,
 	.memory_pressure	= &tcp_memory_pressure,
Signature · 1 line
2.26.2
#5 lore raw

[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc

Quoted 8 lines · Florian Westphal

If memory is tight, tcp_sendpages may fail because it can't allocate the skb head.

Now that we pass DONTWAIT to tcp_sendpages it won't block anymore on the subflow socket when this happens.

We can avoid such allocation failure in tcp code by preallocating an skb skb when we pick the subflow to use.

The sk_tx_skb_cache usage caused some TCP regression upstream:

commit 0b7d7f6b22084a3156f267c85303908a8f4c9a08

Author: Eric Dumazet <edumazet(a)google.com>
Date:   Fri Jun 14 16:22:20 2019 -0700

tcp: add tcp_tx_skb_cache sysctl

but that one was not very clear and I guess does not matter for MPTCP - lacking an established baseline.

So all good ;)

Quoted 5 lines
net/mptcp/protocol.c +1 -0
@@ -656,9 +677,13 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 {
 	struct mptcp_subflow_context *subflow;
+	struct sock *sk = (struct sock *)msk;

net/mptcp/protocol.c static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk, · line 689

Quoted 10 lines
net/mptcp/protocol.c +4 -1
 	struct sock *backup = NULL;
 
-	sock_owned_by_me((const struct sock *)msk);
+	sock_owned_by_me(sk);
+
+	if (!mptcp_sendmsg_alloc_skb(sk))
+		return NULL;
 
 	mptcp_for_each_subflow(msk, subflow) {
 		struct sock *ssk = mptcp_subflow_tcp_sock(subflow);

Don't we need the same for mptcp_subflow_get_retrans() ?

Thanks!

/P

author
#6 lore raw

[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc

Quoted 36 lines · Paolo Abeni
Quoted 8 lines · Florian Westphal

If memory is tight, tcp_sendpages may fail because it can't allocate the skb head.

Now that we pass DONTWAIT to tcp_sendpages it won't block anymore on the subflow socket when this happens.

We can avoid such allocation failure in tcp code by preallocating an skb skb when we pick the subflow to use.

The sk_tx_skb_cache usage caused some TCP regression upstream:

commit 0b7d7f6b22084a3156f267c85303908a8f4c9a08

Author: Eric Dumazet <edumazet(a)google.com>
Date:   Fri Jun 14 16:22:20 2019 -0700

tcp: add tcp_tx_skb_cache sysctl

but that one was not very clear and I guess does not matter for MPTCP - lacking an established baseline.

So all good ;)

Quoted 13 lines
net/mptcp/protocol.c +5 -1
@@ -656,9 +677,13 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 {
 	struct mptcp_subflow_context *subflow;
+	struct sock *sk = (struct sock *)msk;
 	struct sock *backup = NULL;
 
-	sock_owned_by_me((const struct sock *)msk);
+	sock_owned_by_me(sk);
+
+	if (!mptcp_sendmsg_alloc_skb(sk))
+		return NULL;
 
Quoted 4 lines

net/mptcp/protocol.c static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk, · line 689

Quoted 2 lines
net/mptcp/protocol.c +0 -0
 	mptcp_for_each_subflow(msk, subflow) {
 		struct sock *ssk = mptcp_subflow_tcp_sock(subflow);

Don't we need the same for mptcp_subflow_get_retrans() ?

Its optional. If allocation doesn't fail there is no difference.

If it fails then as-is, mptcp_sendmsg_frag will return -EAGAIN and no packet is sent.

If I'd add mptcp_sendmsg_alloc_skb() call to the worker, then it would bail our right before (and would not sleep either).

So the end result is the same: no retransmission takes place.

Let me know if you prefer explicit mptcp_sendmsg_alloc_skb() for retransmit case too.

Thanks!

#7 lore raw

[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc

Quoted 47 lines · Florian Westphal
Quoted 36 lines · Paolo Abeni
Quoted 8 lines · Florian Westphal

If memory is tight, tcp_sendpages may fail because it can't allocate the skb head.

Now that we pass DONTWAIT to tcp_sendpages it won't block anymore on the subflow socket when this happens.

We can avoid such allocation failure in tcp code by preallocating an skb skb when we pick the subflow to use.

The sk_tx_skb_cache usage caused some TCP regression upstream:

commit 0b7d7f6b22084a3156f267c85303908a8f4c9a08

Author: Eric Dumazet <edumazet(a)google.com>
Date:   Fri Jun 14 16:22:20 2019 -0700

tcp: add tcp_tx_skb_cache sysctl

but that one was not very clear and I guess does not matter for MPTCP - lacking an established baseline.

So all good ;)

Quoted 13 lines
net/mptcp/protocol.c +5 -1
@@ -656,9 +677,13 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 {
 	struct mptcp_subflow_context *subflow;
+	struct sock *sk = (struct sock *)msk;
 	struct sock *backup = NULL;
 
-	sock_owned_by_me((const struct sock *)msk);
+	sock_owned_by_me(sk);
+
+	if (!mptcp_sendmsg_alloc_skb(sk))
+		return NULL;
 
Quoted 4 lines

net/mptcp/protocol.c static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk, · line 689

Quoted 2 lines
net/mptcp/protocol.c +0 -0
 	mptcp_for_each_subflow(msk, subflow) {
 		struct sock *ssk = mptcp_subflow_tcp_sock(subflow);

Don't we need the same for mptcp_subflow_get_retrans() ?

Its optional. If allocation doesn't fail there is no difference.

If it fails then as-is, mptcp_sendmsg_frag will return -EAGAIN and no packet is sent.

Quoted 7 lines

If I'd add mptcp_sendmsg_alloc_skb() call to the worker, then it would bail our right before (and would not sleep either).

So the end result is the same: no retransmission takes place.

Let me know if you prefer explicit mptcp_sendmsg_alloc_skb() for retransmit case too.

Ok, I missed/forgot that the retransmission must be non blocking.

I now see that retransmissions are fine as is!

Still on this, don't we need to check mptcp_sendmsg_alloc_skb() inside the sendmsg() loop, too? Elsewhere we could fail on large send if we complete the first skb, keep looping in sendmsg() and the next iteration would block. Am I missing something?

Thanks!

Paolo

author
#8 lore raw

[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc

Quoted 8 lines · Paolo Abeni

Ok, I missed/forgot that the retransmission must be non blocking.

I now see that retransmissions are fine as is!

Still on this, don't we need to check mptcp_sendmsg_alloc_skb() inside the sendmsg() loop, too? Elsewhere we could fail on large send if we complete the first skb, keep looping in sendmsg() and the next iteration would block. Am I missing something?

You mean:

net/mptcp/protocol.c +1 -0
index 780d873b6c19..66a67b11d01b 100644
@@ -814,6 +814,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
                if (!tx_ok)
                        break;
                if (!sk_stream_memory_free(ssk) ||
+                   !mptcp_sendmsg_alloc_skb(ssk) ||
                    !mptcp_page_frag_refill(ssk, pfrag) ||
                    !mptcp_ext_cache_refill(msk)) {
                        tcp_push(ssk, msg->msg_flags, mss_now,

?

Yes, that could be added. Its not a huge deal since mptcp_sendmsg_frag() would yield -EAGAIN if do_tcp_sendpages would block, and that will then re-start for the 'blocking io' case (i.e., ssk is released and 'goto restart' fetches next ssk with posssible wait_for_memory).

#9 lore raw

[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc

Quoted 25 lines · Florian Westphal
Quoted 8 lines · Paolo Abeni

Ok, I missed/forgot that the retransmission must be non blocking.

I now see that retransmissions are fine as is!

Still on this, don't we need to check mptcp_sendmsg_alloc_skb() inside the sendmsg() loop, too? Elsewhere we could fail on large send if we complete the first skb, keep looping in sendmsg() and the next iteration would block. Am I missing something?

You mean:

net/mptcp/protocol.c +1 -0
index 780d873b6c19..66a67b11d01b 100644
@@ -814,6 +814,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
                if (!tx_ok)
                        break;
                if (!sk_stream_memory_free(ssk) ||
+                   !mptcp_sendmsg_alloc_skb(ssk) ||
                    !mptcp_page_frag_refill(ssk, pfrag) ||
                    !mptcp_ext_cache_refill(msk)) {
                        tcp_push(ssk, msg->msg_flags, mss_now,
Quoted 1 line

?

Yep exactly!

Quoted 4 lines

Yes, that could be added. Its not a huge deal since mptcp_sendmsg_frag() would yield -EAGAIN if do_tcp_sendpages would block, and that will then re-start for the 'blocking io' case (i.e., ssk is released and 'goto restart' fetches next ssk with posssible wait_for_memory).

Again, I missed some pieces of the puzzle ;) Yep, we are good even without that. The lack of simmetry WRT the other allocation is a bit confusing to me, but it's really a subjective matter.

I'm good with or without the above chunk

Thanks for the clarifications!

Paolo

(ack to the whole series!)

author
#10 lore raw

PATCH 4/7 mptcp: fill skb extension cache outside of mptcp_sendmsg_frag

In reply to Florian Westphal

The mptcp_sendmsg_frag helper contains a loop that will wait on the subflow sk.

It seems preferrable to only wait in mptcp_sendmsg() when blocking io is requested. mptcp_sendmsg already has such a wait loop that is used when no subflow socket is available for transmission.

This is a preparation patch that makes sure we call mptcp_sendmsg_frag only if a skb extension has been allocated.

Moreover, such allocation currently uses GFP_ATOMIC while it could use sleeping allocation instead.

Followup patches will remove the wait loop from mptcp_sendmsg_frag() and will allow to do a sleeping allocation for the extension.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)
net/mptcp/protocol.c +14 -4
index f28c131c010e..6b3141ecfced 100644
@@ -685,6 +685,9 @@ static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 	if (!mptcp_sendmsg_alloc_skb(sk))
 		return NULL;
 
+	if (!mptcp_ext_cache_refill(msk))
+		return NULL;
+
 	mptcp_for_each_subflow(msk, subflow) {
 		struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
 
@@ -818,7 +821,8 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		tx_ok = msg_data_left(msg);
 		if (!tx_ok)
 			break;
-		if (!sk_stream_memory_free(ssk)) {
+		if (!sk_stream_memory_free(ssk) ||
+		    !mptcp_ext_cache_refill(msk)) {
 			tcp_push(ssk, msg->msg_flags, mss_now,
 				 tcp_sk(ssk)->nonagle, size_goal);
 			release_sock(ssk);
@@ -1139,7 +1143,7 @@ static void mptcp_worker(struct work_struct *work)
 {
 	struct mptcp_sock *msk = container_of(work, struct mptcp_sock, work);
 	struct sock *ssk, *sk = &msk->sk.icsk_inet.sk;
-	int orig_len, orig_offset, ret, mss_now = 0, size_goal = 0;
+	int orig_len, orig_offset, mss_now = 0, size_goal = 0;
 	struct mptcp_data_frag *dfrag;
 	u64 orig_write_seq;
 	size_t copied = 0;
@@ -1161,6 +1165,9 @@ static void mptcp_worker(struct work_struct *work)
 	if (!dfrag)
 		goto unlock;
 
+	if (!mptcp_ext_cache_refill(msk))
+		goto reset_unlock;
+
 	ssk = mptcp_subflow_get_retrans(msk);
 	if (!ssk)
 		goto reset_unlock;
@@ -1172,8 +1179,8 @@ static void mptcp_worker(struct work_struct *work)
 	orig_offset = dfrag->offset;
 	orig_write_seq = dfrag->data_seq;
 	while (dfrag->data_len > 0) {
-		ret = mptcp_sendmsg_frag(sk, ssk, &msg, dfrag, &timeo, &mss_now,
-					 &size_goal);
+		int ret = mptcp_sendmsg_frag(sk, ssk, &msg, dfrag, &timeo,
+					     &mss_now, &size_goal);
 		if (ret < 0)
 			break;
 
@@ -1181,6 +1188,9 @@ static void mptcp_worker(struct work_struct *work)
 		copied += ret;
 		dfrag->data_len -= ret;
 		dfrag->offset += ret;
+
+		if (!mptcp_ext_cache_refill(msk))
+			break;
 	}
 	if (copied)
 		tcp_push(ssk, msg.msg_flags, mss_now, tcp_sk(ssk)->nonagle,
Signature · 1 line
2.26.2
author
#11 lore raw

PATCH 5/7 mptcp: fill skb page frag cache outside of mptcp_sendmsg_frag

In reply to Florian Westphal

The mptcp_sendmsg_frag helper contains a loop that will wait on the subflow sk.

It seems preferrable to only wait in mptcp_sendmsg() when blocking io is requested. mptcp_sendmsg already has such a wait loop that is used when no subflow socket is available for transmission.

This is another preparation patch that makes sure we call mptcp_sendmsg_frag only if the page frag cache has been refilled.

Followup patch will remove the wait loop from mptcp_sendmsg_frag().

The retransmit worker doesn't need to do this refill as it won't transmit new mptcp-level data.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +6 -1
index 6b3141ecfced..06c77f16bf38 100644
@@ -739,6 +739,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 {
 	int mss_now = 0, size_goal = 0, ret = 0;
 	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct page_frag *pfrag;
 	struct socket *ssock;
 	size_t copied = 0;
 	struct sock *ssk;
@@ -767,12 +768,15 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		return ret >= 0 ? ret + copied : (copied ? copied : ret);
 	}
 
+	pfrag = sk_page_frag(sk);
 restart:
 	mptcp_clean_una(sk);
 
 	__mptcp_flush_join_list(msk);
 	ssk = mptcp_subflow_get_send(msk);
-	while (!sk_stream_memory_free(sk) || !ssk) {
+	while (!sk_stream_memory_free(sk) ||
+	       !ssk ||
+	       !mptcp_page_frag_refill(ssk, pfrag)) {
 		ret = sk_stream_wait_memory(sk, &timeo);
 		if (ret)
 			goto out;
@@ -822,6 +826,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		if (!tx_ok)
 			break;
 		if (!sk_stream_memory_free(ssk) ||
+		    !mptcp_page_frag_refill(ssk, pfrag) ||
 		    !mptcp_ext_cache_refill(msk)) {
 			tcp_push(ssk, msg->msg_flags, mss_now,
 				 tcp_sk(ssk)->nonagle, size_goal);
Signature · 1 line
2.26.2
author
#12 lore raw

PATCH 6/7 mptcp: remove inner wait loop from mptcp_sendmsg_frag

In reply to Florian Westphal

previous patches made sure we only call into this function when these prerequisites are met, so no need to wait on the subflow socket anymore.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 14 --------------
 1 file changed, 14 deletions(-)
net/mptcp/protocol.c +0 -14
index 06c77f16bf38..5c1fafb24d12 100644
@@ -531,20 +531,6 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
 	 * fooled into a warning if we don't init here
 	 */
 	pfrag = sk_page_frag(sk);
-	while ((!retransmission && !mptcp_page_frag_refill(ssk, pfrag)) ||
-	       !mptcp_ext_cache_refill(msk)) {
-		ret = sk_stream_wait_memory(ssk, timeo);
-		if (ret)
-			return ret;
-
-		/* if sk_stream_wait_memory() sleeps snd_una can change
-		 * significantly, refresh the rtx queue
-		 */
-		mptcp_clean_una(sk);
-
-		if (unlikely(__mptcp_needs_tcp_fallback(msk)))
-			return 0;
-	}
 	if (!retransmission) {
 		write_seq = &msk->write_seq;
 		page = pfrag->page;
Signature · 1 line
2.26.2
author
#13 lore raw

PATCH 7/7 net: allow __skb_ext_alloc to sleep

In reply to Florian Westphal

mptcp calls this from the transmit side, from process context. Allow a sleeping allocation instead of unconditional GFP_ATOMIC.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 include/linux/skbuff.h | 2 +-
 net/core/skbuff.c      | 8 +++++---
 net/mptcp/protocol.c   | 4 +++-
 3 files changed, 9 insertions(+), 5 deletions(-)
include/linux/skbuff.h +1 -1
index 3000c526f552..531843952809 100644
@@ -4165,7 +4165,7 @@ struct skb_ext {
 	char data[] __aligned(8);
 };
 
-struct skb_ext *__skb_ext_alloc(void);
+struct skb_ext *__skb_ext_alloc(gfp_t flags);
 void *__skb_ext_set(struct sk_buff *skb, enum skb_ext_id id,
 		    struct skb_ext *ext);
 void *skb_ext_add(struct sk_buff *skb, enum skb_ext_id id);
net/core/skbuff.c +5 -3
index 1bf0c3d278e7..35a133c6d13b 100644
@@ -6087,13 +6087,15 @@ static void *skb_ext_get_ptr(struct skb_ext *ext, enum skb_ext_id id)
 /**
  * __skb_ext_alloc - allocate a new skb extensions storage
  *
+ * @flags: See kmalloc().
+ *
  * Returns the newly allocated pointer. The pointer can later attached to a
  * skb via __skb_ext_set().
  * Note: caller must handle the skb_ext as an opaque data.
  */
-struct skb_ext *__skb_ext_alloc(void)
+struct skb_ext *__skb_ext_alloc(gfp_t flags)
 {
-	struct skb_ext *new = kmem_cache_alloc(skbuff_ext_cache, GFP_ATOMIC);
+	struct skb_ext *new = kmem_cache_alloc(skbuff_ext_cache, flags);
 
 	if (new) {
 		memset(new->offset, 0, sizeof(new->offset));
@@ -6188,7 +6190,7 @@ void *skb_ext_add(struct sk_buff *skb, enum skb_ext_id id)
 	} else {
 		newoff = SKB_EXT_CHUNKSIZEOF(*new);
 
-		new = __skb_ext_alloc();
+		new = __skb_ext_alloc(GFP_ATOMIC);
 		if (!new)
 			return NULL;
 	}
net/mptcp/protocol.c +3 -1
index 5c1fafb24d12..85e692b9e8cc 100644
@@ -367,8 +367,10 @@ static void mptcp_stop_timer(struct sock *sk)
 
 static bool mptcp_ext_cache_refill(struct mptcp_sock *msk)
 {
+	const struct sock *sk = (const struct sock *)msk;
+
 	if (!msk->cached_ext)
-		msk->cached_ext = __skb_ext_alloc();
+		msk->cached_ext = __skb_ext_alloc(sk->sk_allocation);
 
 	return !!msk->cached_ext;
 }
Signature · 1 line
2.26.2
#14 lore raw
In reply to Florian Westphal
Quoted 16 lines · Florian Westphal

This series reworks mptcp_sendmsg logic to avoid blocking on the subflow socket in an attempt to address/resolve:
https://github.com/multipath-tcp/mptcp_net-next/issues/7

It does so by removing the wait loop from mptcp_sendmsg_frag helper.

In order to do that, it moves prerequisites into the callers.

The worker can just reschedule in case no subflow socket is ready.

The sendmsg path can use the existing wait logic.

Because large send requests can result in multiple mptcp_sendmsg_frag calls from mptcp_sendmsg, we may need to restart the socket lookup in case subflow can't accept more data or memory is low.

Quoted 9 lines

Doing so blocks on the mptcp socket, and existing wait handling releases the msk lock while blocking.

Lastly, no need to use ATOMIC allocations for extension allocation, so allow to pass gfp_t arg to the helper instead of hard-coded ATOMIC in the core.

Comments welcome.

After looking over the rest of the series I don't have other changes to suggest, series looks good. Thanks for the clarification in Paolo's thread.

Mat

Quoted 9 lines

Florian Westphal (7):
mptcp: avoid blocking in tcp_sendpages
mptcp: make sure timer is inited/started once data is pending for transmission
mptcp: avoid blocking in tcp_sendpages due to skb alloc
mptcp: fill skb extension cache outside of mptcp_sendmsg_frag
mptcp: fill skb page frag cache outside of mptcp_sendmsg_frag
mptcp: remove inner wait loop from mptcp_sendmsg_frag
net: allow __skb_ext_alloc to sleep

Quoted 4 lines
include/linux/skbuff.h |   2 +-
net/core/skbuff.c      |   8 ++--
net/mptcp/protocol.c   | 100 ++++++++++++++++++++++++++++++++++++-------------
3 files changed, 81 insertions(+), 29 deletions(-)

--
Mat Martineau
Intel