mptcp: don't sleep on ssk with msk lock held
Subsystems (2)
Florian Westphal
cover mptcp: don't sleep on ssk with msk lock held
This series reworks mptcp_sendmsg logic to avoid blocking on the subflow socket in an attempt to address/resolve:
https://github.com/multipath-tcp/mptcp_net-next/issues/7
It does so by removing the wait loop from mptcp_sendmsg_frag helper.
In order to do that, it moves prerequisites into the callers.
The worker can just reschedule in case no subflow socket is ready.
The sendmsg path can use the existing wait logic.
Because large send requests can result in multiple mptcp_sendmsg_frag calls from mptcp_sendmsg, we may need to restart the socket lookup in case subflow can't accept more data or memory is low.
Doing so blocks on the mptcp socket, and existing wait handling releases the msk lock while blocking.
Lastly, no need to use ATOMIC allocations for extension allocation, so allow to pass gfp_t arg to the helper instead of hard-coded ATOMIC in the core.
Comments welcome.
Florian Westphal (7):
mptcp: avoid blocking in tcp_sendpages
mptcp: make sure timer is inited/started once data is pending for transmission
mptcp: avoid blocking in tcp_sendpages due to skb alloc
mptcp: fill skb extension cache outside of mptcp_sendmsg_frag
mptcp: fill skb page frag cache outside of mptcp_sendmsg_frag
mptcp: remove inner wait loop from mptcp_sendmsg_frag
net: allow __skb_ext_alloc to sleep
include/linux/skbuff.h | 2 +- net/core/skbuff.c | 8 ++-- net/mptcp/protocol.c | 100 ++++++++++++++++++++++++++++++++++++------------- 3 files changed, 81 insertions(+), 29 deletions(-)
Florian Westphal
PATCH 1/7 mptcp: avoid blocking in tcp_sendpages
The transmit loop continues to xmit new data until an error is returned or all data was transmitted.
For the blocking i/o case, this means that tcp_sendpages() may block on the subflow until more space becomes available, i.e. we end up sleeping with the mptcp socket lock held.
Instead we should check if a different subflow is ready to be used.
This restarts the subflow sk lookup when the tx operation succeeded and the tcp subflow can't accept more data or if tcp_sendpages indicates -EAGAIN on a blocking mptcp socket.
In case all subflows are busy, the existing logic will wait until a subflow is ready, releasing the mptcp level socket while doing so.
The mptcp worker already sets DONTWAIT, so no need to make changes there.
--- net/mptcp/protocol.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
net/mptcp/protocol.c +20 -3
@@ -590,7 +590,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,* access the skb after the sendpages call */ ret =do_tcp_sendpages (ssk, page, offset, psize, - msg->msg_flags |MSG_SENDPAGE_NOTLAST ); + msg->msg_flags |MSG_SENDPAGE_NOTLAST |MSG_DONTWAIT );if (ret <=0 )return ret;
@@ -714,6 +714,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)struct socket *ssock;size_t copied =0 ;struct sock *ssk; +bool tx_ok;long timeo;if (msg->msg_flags & ~(MSG_MORE |MSG_DONTWAIT |MSG_NOSIGNAL ))
@@ -738,6 +739,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)return ret >=0 ? ret + copied : (copied ? copied : ret); } +restart:mptcp_clean_una (sk);__mptcp_flush_join_list (msk);
@@ -759,11 +761,17 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)pr_debug ("conn_list->subflow=%p" , ssk);lock_sock (ssk); -while (msg_data_left (msg)) { + tx_ok =msg_data_left (msg); +while (tx_ok) { ret =mptcp_sendmsg_frag (sk, ssk, msg,NULL , &timeo, &mss_now, &size_goal); -if (ret <0 ) +if (ret <0 ) { +if (ret == -EAGAIN && timeo >0 ) { +release_sock (ssk); + goto restart; + }break ; + }if (ret ==0 &&unlikely (__mptcp_needs_tcp_fallback (msk))) {/* Can happen for passive sockets: * 3WHS negotiated MPTCP, but first packet after is
@@ -777,6 +785,15 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } copied += ret; + tx_ok =msg_data_left (msg); +if (!tx_ok) +break ; +if (!sk_stream_memory_free (ssk)) { +tcp_push (ssk, msg->msg_flags, mss_now, +tcp_sk (ssk)->nonagle, size_goal); +release_sock (ssk); + goto restart; + } }mptcp_set_timeout (sk, ssk);
Signature · 1 line
2.26.2
Mat Martineau
[MPTCP] Re: [PATCH mptcp-next 1/7] mptcp: avoid blocking in tcp_sendpages
Quoted 9 lines · Florian Westphal
The transmit loop continues to xmit new data until an error is returned or all data was transmitted.
For the blocking i/o case, this means that
tcp_sendpages()may block on the subflow until more space becomes available, i.e. we end up sleeping with the mptcp socket lock held.Instead we should check if a different subflow is ready to be used.
Keep in mind that configurable packet schedulers (when eventually implemented) may want to make other decisions when one subflow gets blocked, even if other subflows have available space. One specific subflow might be favored due to better latency, or a redundant scheduler could send the same data on all the available subflows.
I think this patch set makes it easier to make mptcp_subflow_get_send configurable and to have the scheduler reselect when a subflow is out of space, so that's good!
Mat
Quoted 87 lines
This restarts the subflow sk lookup when the tx operation succeeded and the tcp subflow can't accept more data or if
tcp_sendpagesindicates-EAGAINon a blocking mptcp socket.In case all subflows are busy, the existing logic will wait until a subflow is ready, releasing the mptcp level socket while doing so.
The mptcp worker already sets DONTWAIT, so no need to make changes there.
Reported-by Paolo Abeni <pabeni(a)redhat.com>Signed-off-by Florian Westphal <fw(a)strlen.de>---
net/mptcp/protocol.c| 23 ++++++++++++++++++++---1 file changed, 20 insertions(+), 3 deletions(-)net/mptcp/protocol.c +20
-3@@ -590,7 +590,7 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,* access the skb after the sendpages call */ ret =do_tcp_sendpages (ssk, page, offset, psize, - msg->msg_flags |MSG_SENDPAGE_NOTLAST ); + msg->msg_flags |MSG_SENDPAGE_NOTLAST |MSG_DONTWAIT );if (ret <=0 )return ret;@@ -714,6 +714,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)struct socket *ssock;size_t copied =0 ;struct sock *ssk; +bool tx_ok;long timeo;if (msg->msg_flags & ~(MSG_MORE |MSG_DONTWAIT |MSG_NOSIGNAL ))@@ -738,6 +739,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)return ret >=0 ? ret + copied : (copied ? copied : ret); } +restart:mptcp_clean_una (sk);__mptcp_flush_join_list (msk);@@ -759,11 +761,17 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)pr_debug ("conn_list->subflow=%p" , ssk);lock_sock (ssk); -while (msg_data_left (msg)) { + tx_ok =msg_data_left (msg); +while (tx_ok) { ret =mptcp_sendmsg_frag (sk, ssk, msg,NULL , &timeo, &mss_now, &size_goal); -if (ret <0 ) +if (ret <0 ) { +if (ret == -EAGAIN && timeo >0 ) { +release_sock (ssk); + goto restart; + }break ; + }if (ret ==0 &&unlikely (__mptcp_needs_tcp_fallback (msk))) {/* Can happen for passive sockets: * 3WHS negotiated MPTCP, but first packet after is @@ -777,6 +785,15 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } copied += ret; + tx_ok =msg_data_left (msg); +if (!tx_ok) +break ; +if (!sk_stream_memory_free (ssk)) { +tcp_push (ssk, msg->msg_flags, mss_now, +tcp_sk (ssk)->nonagle, size_goal); +release_sock (ssk); + goto restart; + } }mptcp_set_timeout (sk, ssk);Signature · 4 lines
2.26.2 _______________________________________________ mptcp mailing list -- mptcp(a)lists.01.org To unsubscribe send an email to mptcp-leave(a)lists.01.org
--
Mat Martineau
Intel
Florian Westphal
PATCH 2/7 mptcp: make sure timer is inited/started once data is pending for transmission
In reply to Florian WestphalThe mptcp_connect self-tests with additional '-m mmap' argument triggers a (harmless) WARN_ON_ONCE splat in mptcp_reset_timer() as the mptcp timer is not initialized yet.
This is because the timer is only set up after the transmit loop, but consecutive iterations can trigger the splat.
Make sure the timer is set as soon as we pass the first chunk of data.
--- net/mptcp/protocol.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
net/mptcp/protocol.c +5 -2
@@ -785,6 +785,11 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } copied += ret; +mptcp_set_timeout (sk, ssk); +/* start the timer, if it's not pending */ +if (!mptcp_timer_pending (sk)) +mptcp_reset_timer (sk); + tx_ok =msg_data_left (msg);if (!tx_ok)break ;
@@ -796,12 +801,10 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } } -mptcp_set_timeout (sk, ssk);if (copied) { ret = copied;tcp_push (ssk, msg->msg_flags, mss_now,tcp_sk (ssk)->nonagle, size_goal); -/* start the timer, if it's not pending */ if (!mptcp_timer_pending (sk))mptcp_reset_timer (sk);
Signature · 1 line
2.26.2
Florian Westphal
PATCH 3/7 mptcp: avoid blocking in tcp_sendpages due to skb alloc
In reply to Florian WestphalIf memory is tight, tcp_sendpages may fail because it can't allocate the skb head.
Now that we pass DONTWAIT to tcp_sendpages it won't block anymore on the subflow socket when this happens.
We can avoid such allocation failure in tcp code by preallocating an skb skb when we pick the subflow to use.
If it fails, we can wait on the mptcp socket until memory becomes available.
--- net/mptcp/protocol.c | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +28 -1
@@ -488,6 +488,22 @@ mptcp_carve_data_frag(const struct mptcp_sock *msk, struct page_frag *pfrag,return dfrag; } +static bool mptcp_sendmsg_alloc_skb (struct sock *sk) +{ +struct sk_buff *skb; + +if (sk->sk_tx_skb_cache) +return true; + + skb =alloc_skb_fclone (sk->sk_prot->max_header, sk->sk_allocation); +if (skb) { +skb_reserve (skb, sk->sk_prot->max_header); + sk->sk_tx_skb_cache = skb; + } + +return sk->sk_tx_skb_cache !=NULL ; +} + static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk, struct msghdr *msg, struct mptcp_data_frag *dfrag, long *timeo, int *pmss_now,
@@ -504,6 +520,11 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,u64 *write_seq;size_t psize; +if (!ssk->sk_tx_skb_cache) { + ssk->sk_tx_skb_cache = sk->sk_tx_skb_cache; + sk->sk_tx_skb_cache =NULL ; + } +/* use the mptcp page cache so that we can easily move the data * from one substream to another, but do per subflow memory accounting * Note: pfrag is used only !retransmission, but the compiler if
@@ -656,9 +677,13 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,static struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk) {struct mptcp_subflow_context *subflow; +struct sock *sk = (struct sock *)msk;struct sock *backup =NULL ; -sock_owned_by_me ((const struct sock *)msk); +sock_owned_by_me (sk); + +if (!mptcp_sendmsg_alloc_skb (sk)) +return NULL ;mptcp_for_each_subflow (msk, subflow) {struct sock *ssk =mptcp_subflow_tcp_sock (subflow);
@@ -1478,6 +1503,7 @@ static void mptcp_destroy(struct sock *sk)if (msk->cached_ext)__skb_ext_put (msk->cached_ext); +kfree_skb (sk->sk_tx_skb_cache);sk_sockets_allocated_dec (sk); }
@@ -1688,6 +1714,7 @@ static struct proto mptcp_prot = { .hash = inet_hash, .unhash = inet_unhash, .get_port = mptcp_get_port, + .max_header = MAX_TCP_HEADER, .sockets_allocated = &mptcp_sockets_allocated, .memory_allocated = &tcp_memory_allocated, .memory_pressure = &tcp_memory_pressure,
Signature · 1 line
2.26.2
Paolo Abeni
[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc
Quoted 8 lines · Florian Westphal
If memory is tight,
tcp_sendpagesmay fail because it can't allocate the skb head.Now that we pass DONTWAIT to
tcp_sendpagesit won't block anymore on the subflow socket when this happens.We can avoid such allocation failure in tcp code by preallocating an skb skb when we pick the subflow to use.
The sk_tx_skb_cache usage caused some TCP regression upstream:
commit 0b7d7f6b22084a3156f267c85303908a8f4c9a08
Author: Eric Dumazet <edumazet(a)google.com> Date: Fri Jun 14 16:22:20 2019 -0700
tcp: add tcp_tx_skb_cache sysctl
but that one was not very clear and I guess does not matter for MPTCP - lacking an established baseline.
So all good ;)
Quoted 5 lines
net/mptcp/protocol.c +1
-0@@ -656,9 +677,13 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,static struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk) {struct mptcp_subflow_context *subflow; +struct sock *sk = (struct sock *)msk;
net/mptcp/protocol.c static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk, · line 689
Quoted 10 lines
net/mptcp/protocol.c +4
-1struct sock *backup =NULL ; -sock_owned_by_me ((const struct sock *)msk); +sock_owned_by_me (sk); + +if (!mptcp_sendmsg_alloc_skb (sk)) +return NULL ;mptcp_for_each_subflow (msk, subflow) {struct sock *ssk =mptcp_subflow_tcp_sock (subflow);
Don't we need the same for mptcp_subflow_get_retrans() ?
Thanks!
/P
Florian Westphal
[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc
Quoted 36 lines · Paolo Abeni
Quoted 8 lines · Florian Westphal
If memory is tight,
tcp_sendpagesmay fail because it can't allocate the skb head.Now that we pass DONTWAIT to
tcp_sendpagesit won't block anymore on the subflow socket when this happens.We can avoid such allocation failure in tcp code by preallocating an skb skb when we pick the subflow to use.
The
sk_tx_skb_cacheusage caused some TCP regression upstream:commit
0b7d7f6b22084a3156f267c85303908a8f4c9a08Author: Eric Dumazet <edumazet(a)google.com> Date: Fri Jun 14 16:22:20 2019 -0700tcp: add
tcp_tx_skb_cachesysctlbut that one was not very clear and I guess does not matter for MPTCP - lacking an established baseline.
So all good ;)
Quoted 13 lines
net/mptcp/protocol.c +5
-1@@ -656,9 +677,13 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,static struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk) {struct mptcp_subflow_context *subflow; +struct sock *sk = (struct sock *)msk;struct sock *backup =NULL ; -sock_owned_by_me ((const struct sock *)msk); +sock_owned_by_me (sk); + +if (!mptcp_sendmsg_alloc_skb (sk)) +return NULL ;
Quoted 4 lines
net/mptcp/protocol.c static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk, · line 689
Quoted 2 lines
net/mptcp/protocol.c +0
-0mptcp_for_each_subflow (msk, subflow) {struct sock *ssk =mptcp_subflow_tcp_sock (subflow);Don't we need the same for
mptcp_subflow_get_retrans()?
Its optional. If allocation doesn't fail there is no difference.
If it fails then as-is, mptcp_sendmsg_frag will return -EAGAIN and no packet is sent.
If I'd add mptcp_sendmsg_alloc_skb() call to the worker, then it would bail our right before (and would not sleep either).
So the end result is the same: no retransmission takes place.
Let me know if you prefer explicit mptcp_sendmsg_alloc_skb() for retransmit case too.
Thanks!
Paolo Abeni
[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc
Quoted 47 lines · Florian Westphal
Quoted 36 lines · Paolo Abeni
Quoted 8 lines · Florian Westphal
If memory is tight,
tcp_sendpagesmay fail because it can't allocate the skb head.Now that we pass DONTWAIT to
tcp_sendpagesit won't block anymore on the subflow socket when this happens.We can avoid such allocation failure in tcp code by preallocating an skb skb when we pick the subflow to use.
The
sk_tx_skb_cacheusage caused some TCP regression upstream:commit
0b7d7f6b22084a3156f267c85303908a8f4c9a08Author: Eric Dumazet <edumazet(a)google.com> Date: Fri Jun 14 16:22:20 2019 -0700tcp: add
tcp_tx_skb_cachesysctlbut that one was not very clear and I guess does not matter for MPTCP - lacking an established baseline.
So all good ;)
Quoted 13 lines
net/mptcp/protocol.c +5
-1@@ -656,9 +677,13 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,static struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk) {struct mptcp_subflow_context *subflow; +struct sock *sk = (struct sock *)msk;struct sock *backup =NULL ; -sock_owned_by_me ((const struct sock *)msk); +sock_owned_by_me (sk); + +if (!mptcp_sendmsg_alloc_skb (sk)) +return NULL ;Quoted 4 lines
net/mptcp/protocol.c static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk, · line 689
Quoted 2 lines
net/mptcp/protocol.c +0
-0mptcp_for_each_subflow (msk, subflow) {struct sock *ssk =mptcp_subflow_tcp_sock (subflow);Don't we need the same for
mptcp_subflow_get_retrans()?Its optional. If allocation doesn't fail there is no difference.
If it fails then as-is,
mptcp_sendmsg_fragwill return-EAGAINand no packet is sent.
Quoted 7 lines
If I'd add
mptcp_sendmsg_alloc_skb()call to the worker, then it would bail our right before (and would not sleep either).So the end result is the same: no retransmission takes place.
Let me know if you prefer explicit
mptcp_sendmsg_alloc_skb()for retransmit case too.
Ok, I missed/forgot that the retransmission must be non blocking.
I now see that retransmissions are fine as is!
Still on this, don't we need to check mptcp_sendmsg_alloc_skb() inside the sendmsg() loop, too? Elsewhere we could fail on large send if we complete the first skb, keep looping in sendmsg() and the next iteration would block. Am I missing something?
Thanks!
Paolo
Florian Westphal
[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc
Quoted 8 lines · Paolo Abeni
Ok, I missed/forgot that the retransmission must be non blocking.
I now see that retransmissions are fine as is!
Still on this, don't we need to check
mptcp_sendmsg_alloc_skb()inside thesendmsg()loop, too? Elsewhere we could fail on large send if we complete the first skb, keep looping insendmsg()and the next iteration would block. Am I missing something?
You mean:
net/mptcp/protocol.c +1 -0
@@ -814,6 +814,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)if (!tx_ok)break ;if (!sk_stream_memory_free (ssk) || + !mptcp_sendmsg_alloc_skb (ssk) || !mptcp_page_frag_refill (ssk, pfrag) || !mptcp_ext_cache_refill (msk)) { tcp_push(ssk, msg->msg_flags, mss_now,
?
Yes, that could be added. Its not a huge deal since mptcp_sendmsg_frag() would yield -EAGAIN if do_tcp_sendpages would block, and that will then re-start for the 'blocking io' case (i.e., ssk is released and 'goto restart' fetches next ssk with posssible wait_for_memory).
Paolo Abeni
[MPTCP] Re: [PATCH mptcp-next 3/7] mptcp: avoid blocking in tcp_sendpages due to skb alloc
Quoted 25 lines · Florian Westphal
Quoted 8 lines · Paolo Abeni
Ok, I missed/forgot that the retransmission must be non blocking.
I now see that retransmissions are fine as is!
Still on this, don't we need to check
mptcp_sendmsg_alloc_skb()inside thesendmsg()loop, too? Elsewhere we could fail on large send if we complete the first skb, keep looping insendmsg()and the next iteration would block. Am I missing something?You mean:
net/mptcp/protocol.c +1
-0@@ -814,6 +814,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)if (!tx_ok)break ;if (!sk_stream_memory_free (ssk) || + !mptcp_sendmsg_alloc_skb (ssk) || !mptcp_page_frag_refill (ssk, pfrag) || !mptcp_ext_cache_refill (msk)) { tcp_push(ssk, msg->msg_flags, mss_now,
Quoted 1 line
?
Yep exactly!
Quoted 4 lines
Yes, that could be added. Its not a huge deal since
mptcp_sendmsg_frag()would yield-EAGAINifdo_tcp_sendpageswould block, and that will then re-start for the 'blocking io' case (i.e., ssk is released and 'goto restart' fetches next ssk with posssiblewait_for_memory).
Again, I missed some pieces of the puzzle ;) Yep, we are good even without that. The lack of simmetry WRT the other allocation is a bit confusing to me, but it's really a subjective matter.
I'm good with or without the above chunk
Thanks for the clarifications!
Paolo
(ack to the whole series!)
Florian Westphal
PATCH 4/7 mptcp: fill skb extension cache outside of mptcp_sendmsg_frag
In reply to Florian WestphalThe mptcp_sendmsg_frag helper contains a loop that will wait on the subflow sk.
It seems preferrable to only wait in mptcp_sendmsg() when blocking io is requested. mptcp_sendmsg already has such a wait loop that is used when no subflow socket is available for transmission.
This is a preparation patch that makes sure we call mptcp_sendmsg_frag only if a skb extension has been allocated.
Moreover, such allocation currently uses GFP_ATOMIC while it could use sleeping allocation instead.
Followup patches will remove the wait loop from mptcp_sendmsg_frag() and will allow to do a sleeping allocation for the extension.
--- net/mptcp/protocol.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
net/mptcp/protocol.c +14 -4
@@ -685,6 +685,9 @@ static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)if (!mptcp_sendmsg_alloc_skb (sk))return NULL ; +if (!mptcp_ext_cache_refill (msk)) +return NULL ; +mptcp_for_each_subflow (msk, subflow) {struct sock *ssk =mptcp_subflow_tcp_sock (subflow);
@@ -818,7 +821,8 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) tx_ok =msg_data_left (msg);if (!tx_ok)break ; -if (!sk_stream_memory_free (ssk)) { +if (!sk_stream_memory_free (ssk) || + !mptcp_ext_cache_refill (msk)) {tcp_push (ssk, msg->msg_flags, mss_now,tcp_sk (ssk)->nonagle, size_goal);release_sock (ssk);
@@ -1139,7 +1143,7 @@ static void mptcp_worker(struct work_struct *work) { struct mptcp_sock *msk = container_of(work, struct mptcp_sock, work);struct sock *ssk, *sk = &msk->sk.icsk_inet.sk; -int orig_len, orig_offset, ret, mss_now =0 , size_goal =0 ; +int orig_len, orig_offset, mss_now =0 , size_goal =0 ;struct mptcp_data_frag *dfrag;u64 orig_write_seq;size_t copied =0 ;
@@ -1161,6 +1165,9 @@ static void mptcp_worker(struct work_struct *work)if (!dfrag) goto unlock; +if (!mptcp_ext_cache_refill (msk)) + goto reset_unlock; + ssk =mptcp_subflow_get_retrans (msk);if (!ssk) goto reset_unlock;
@@ -1172,8 +1179,8 @@ static void mptcp_worker(struct work_struct *work) orig_offset = dfrag->offset; orig_write_seq = dfrag->data_seq;while (dfrag->data_len >0 ) { - ret =mptcp_sendmsg_frag (sk, ssk, &msg, dfrag, &timeo, &mss_now, - &size_goal); +int ret =mptcp_sendmsg_frag (sk, ssk, &msg, dfrag, &timeo, + &mss_now, &size_goal);if (ret <0 )break ;
@@ -1181,6 +1188,9 @@ static void mptcp_worker(struct work_struct *work) copied += ret; dfrag->data_len -= ret; dfrag->offset += ret; + +if (!mptcp_ext_cache_refill (msk)) +break ; }if (copied) tcp_push(ssk, msg.msg_flags, mss_now, tcp_sk(ssk)->nonagle,
Signature · 1 line
2.26.2
Florian Westphal
PATCH 5/7 mptcp: fill skb page frag cache outside of mptcp_sendmsg_frag
In reply to Florian WestphalThe mptcp_sendmsg_frag helper contains a loop that will wait on the subflow sk.
It seems preferrable to only wait in mptcp_sendmsg() when blocking io is requested. mptcp_sendmsg already has such a wait loop that is used when no subflow socket is available for transmission.
This is another preparation patch that makes sure we call mptcp_sendmsg_frag only if the page frag cache has been refilled.
Followup patch will remove the wait loop from mptcp_sendmsg_frag().
The retransmit worker doesn't need to do this refill as it won't transmit new mptcp-level data.
--- net/mptcp/protocol.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +6 -1
@@ -739,6 +739,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) {int mss_now =0 , size_goal =0 , ret =0 ;struct mptcp_sock *msk =mptcp_sk (sk); +struct page_frag *pfrag;struct socket *ssock;size_t copied =0 ;struct sock *ssk;
@@ -767,12 +768,15 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)return ret >=0 ? ret + copied : (copied ? copied : ret); } + pfrag =sk_page_frag (sk); restart:mptcp_clean_una (sk);__mptcp_flush_join_list (msk); ssk =mptcp_subflow_get_send (msk); -while (!sk_stream_memory_free (sk) || !ssk) { +while (!sk_stream_memory_free (sk) || + !ssk || + !mptcp_page_frag_refill (ssk, pfrag)) { ret =sk_stream_wait_memory (sk, &timeo);if (ret) goto out;
@@ -822,6 +826,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)if (!tx_ok)break ;if (!sk_stream_memory_free (ssk) || + !mptcp_page_frag_refill (ssk, pfrag) || !mptcp_ext_cache_refill (msk)) {tcp_push (ssk, msg->msg_flags, mss_now,tcp_sk (ssk)->nonagle, size_goal);
Signature · 1 line
2.26.2
Florian Westphal
PATCH 6/7 mptcp: remove inner wait loop from mptcp_sendmsg_frag
In reply to Florian Westphalprevious patches made sure we only call into this function when these prerequisites are met, so no need to wait on the subflow socket anymore.
--- net/mptcp/protocol.c | 14 -------------- 1 file changed, 14 deletions(-)
net/mptcp/protocol.c +0 -14
@@ -531,20 +531,6 @@ static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,* fooled into a warning if we don't init here */ pfrag =sk_page_frag (sk); -while ((!retransmission && !mptcp_page_frag_refill (ssk, pfrag)) || - !mptcp_ext_cache_refill (msk)) { - ret =sk_stream_wait_memory (ssk, timeo); -if (ret) -return ret; - -/* if sk_stream_wait_memory() sleeps snd_una can change -* significantly, refresh the rtx queue -*/ -mptcp_clean_una (sk); - -if (unlikely (__mptcp_needs_tcp_fallback (msk))) -return 0 ; - }if (!retransmission) { write_seq = &msk->write_seq; page = pfrag->page;
Signature · 1 line
2.26.2
Florian Westphal
PATCH 7/7 net: allow __skb_ext_alloc to sleep
In reply to Florian Westphalmptcp calls this from the transmit side, from process context. Allow a sleeping allocation instead of unconditional GFP_ATOMIC.
--- include/linux/skbuff.h | 2 +- net/core/skbuff.c | 8 +++++--- net/mptcp/protocol.c | 4 +++- 3 files changed, 9 insertions(+), 5 deletions(-)
include/linux/skbuff.h +1 -1
@@ -4165,7 +4165,7 @@ struct skb_ext {char data[]__aligned(8) ; }; -struct skb_ext *__skb_ext_alloc (void ); +struct skb_ext *__skb_ext_alloc (gfp_t flags);void *__skb_ext_set (struct sk_buff *skb,enum skb_ext_id id,struct skb_ext *ext);void *skb_ext_add (struct sk_buff *skb,enum skb_ext_id id);
net/core/skbuff.c +5 -3
@@ -6087,13 +6087,15 @@ static void *skb_ext_get_ptr(struct skb_ext *ext, enum skb_ext_id id)/** * __skb_ext_alloc - allocate a new skb extensions storage * +* @flags: See kmalloc(). +* * Returns the newly allocated pointer. The pointer can later attached to a * skb via __skb_ext_set(). * Note: caller must handle the skb_ext as an opaque data. */ -struct skb_ext *__skb_ext_alloc (void ) +struct skb_ext *__skb_ext_alloc (gfp_t flags) { -struct skb_ext *new =kmem_cache_alloc (skbuff_ext_cache,GFP_ATOMIC ); +struct skb_ext *new =kmem_cache_alloc (skbuff_ext_cache, flags);if (new) {memset (new->offset,0 ,sizeof (new->offset));
@@ -6188,7 +6190,7 @@ void *skb_ext_add(struct sk_buff *skb, enum skb_ext_id id) }else { newoff =SKB_EXT_CHUNKSIZEOF (*new); - new =__skb_ext_alloc (); + new =__skb_ext_alloc (GFP_ATOMIC );if (!new)return NULL ; }
net/mptcp/protocol.c +3 -1
@@ -367,8 +367,10 @@ static void mptcp_stop_timer(struct sock *sk)static bool mptcp_ext_cache_refill (struct mptcp_sock *msk) { +const struct sock *sk = (const struct sock *)msk; +if (!msk->cached_ext) - msk->cached_ext =__skb_ext_alloc (); + msk->cached_ext =__skb_ext_alloc (sk->sk_allocation);return !!msk->cached_ext; }
Signature · 1 line
2.26.2
Mat Martineau
Quoted 16 lines · Florian Westphal
This series reworks
mptcp_sendmsglogic to avoid blocking on the subflow socket in an attempt to address/resolve:
https://github.com/multipath-tcp/mptcp_net-next/issues/7It does so by removing the wait loop from
mptcp_sendmsg_fraghelper.In order to do that, it moves prerequisites into the callers.
The worker can just reschedule in case no subflow socket is ready.
The sendmsg path can use the existing wait logic.
Because large send requests can result in multiple
mptcp_sendmsg_fragcalls frommptcp_sendmsg, we may need to restart the socket lookup in case subflow can't accept more data or memory is low.
Quoted 9 lines
Doing so blocks on the mptcp socket, and existing wait handling releases the msk lock while blocking.
Lastly, no need to use ATOMIC allocations for extension allocation, so allow to pass
gfp_targ to the helper instead of hard-coded ATOMIC in the core.Comments welcome.
After looking over the rest of the series I don't have other changes to suggest, series looks good. Thanks for the clarification in Paolo's thread.
Mat
Quoted 9 lines
Florian Westphal (7):
mptcp: avoid blocking intcp_sendpages
mptcp: make sure timer is inited/started once data is pending for transmission
mptcp: avoid blocking intcp_sendpagesdue to skb alloc
mptcp: fill skb extension cache outside ofmptcp_sendmsg_frag
mptcp: fill skb page frag cache outside ofmptcp_sendmsg_frag
mptcp: remove inner wait loop frommptcp_sendmsg_frag
net: allow__skb_ext_allocto sleep
Quoted 4 lines
include/linux/skbuff.h | 2 +- net/core/skbuff.c | 8 ++-- net/mptcp/protocol.c | 100 ++++++++++++++++++++++++++++++++++++------------- 3 files changed, 81 insertions(+), 29 deletions(-)
--
Mat Martineau
Intel