mptcp: switch mptcp_poll to use parent sock wait queue
Subsystems (3)
Florian Westphal
v2 cover mptcp: switch mptcp_poll to use parent sock wait queue
No major changes, except the rebase on current export branch. Selftest still passes.
This series is needed for sendmsg refactor, without it we can't make mptcp sendmsg wait on the mptcp socket wait queue because we don't get the needed wakeup events from the subflow layer.
The following changes since commit 85c93de446d3fd90e47605ba8ff161bef7558f09:
mptcp: Implement basic path manager (2019-10-23 01:31:04 +0000)
are available in the Git repository at:
git://git.breakpoint.cc/fw/mptcp-next.git mptcp_poll_07
for you to fetch changes up to 864dd47df1293e4b8bfc70be0d459286b6dddc3a:
mptcp: switch mptcp_poll to mptcp socket wait queue (2019-10-23 15:20:28 +0200)
----------------------------------------------------------------
Florian Westphal (4):
mptcp: add and use subflow_write_space
subflow: wake parent mptcp socket on subflow state change
net: tcp: add __tcp_poll helper
mptcp: switch mptcp_poll to mptcp socket wait queue
include/net/tcp.h | 1 + net/ipv4/tcp.c | 12 ++++++++---- net/mptcp/protocol.c | 4 +++- net/mptcp/subflow.c | 34 ++++++++++++++++++++++++++++++++++ 4 files changed, 46 insertions(+), 5 deletions(-)
Florian Westphal
PATCH v2 1/4 mptcp: add and use subflow_write_space
This will be used to unblock tasks waiting for POLLOUT event once mptcp_poll is altered to not wait for each subflow anymore.
--- net/mptcp/subflow.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
net/mptcp/subflow.c +11 -0
@@ -610,6 +610,16 @@ static void subflow_data_ready(struct sock *sk) } } +static void subflow_write_space (struct sock *sk) +{ +struct mptcp_subflow_context *subflow =mptcp_subflow_ctx (sk); +struct sock *parent = subflow->conn; + +sk_stream_write_space (sk); +if (parent) +sk_stream_write_space (parent); +} +int mptcp_subflow_connect (struct sock *sk,struct sockaddr *local,struct sockaddr *remote,u8 remote_id) {
@@ -739,6 +749,7 @@ static int subflow_ulp_init(struct sock *sk) icsk->icsk_af_ops = &subflow_specific; ctx->tcp_sk_data_ready = sk->sk_data_ready; sk->sk_data_ready = subflow_data_ready; + sk->sk_write_space = subflow_write_space; out:return err; }
Signature · 1 line
2.21.0
Florian Westphal
PATCH v2 2/4 subflow: wake parent mptcp socket on subflow state change
In reply to Florian WestphalWe need to wakeup readers blocking for POLLIN, as no more data will arrive they will never be awoken otherwise.
This is needed at least until proper MPTCP-Level fin/reset signalling gets added.
--- net/mptcp/subflow.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
net/mptcp/subflow.c +23 -0
@@ -722,6 +722,28 @@ static struct mptcp_subflow_context *subflow_create_ctx(struct sock *sk,return ctx; } +static void __subflow_state_change (struct sock *sk) +{ +struct socket_wq *wq; + +rcu_read_lock (); + wq =rcu_dereference (sk->sk_wq); +if (skwq_has_sleeper (wq)) +wake_up_interruptible_all (&wq->wait); +rcu_read_unlock (); +} + +static void subflow_state_change (struct sock *sk) +{ +struct mptcp_subflow_context *subflow =mptcp_subflow_ctx (sk); +struct sock *parent = subflow->conn; + +__subflow_state_change (sk); + +if (parent) +__subflow_state_change (parent); +} +static int subflow_ulp_init (struct sock *sk) {struct inet_connection_sock *icsk =inet_csk (sk);
@@ -750,6 +772,7 @@ static int subflow_ulp_init(struct sock *sk) ctx->tcp_sk_data_ready = sk->sk_data_ready; sk->sk_data_ready = subflow_data_ready; sk->sk_write_space = subflow_write_space; + sk->sk_state_change = subflow_state_change; out:return err; }
Signature · 1 line
2.21.0
Mat Martineau
[MPTCP] Re: [PATCH v2 2/4] subflow: wake parent mptcp socket on subflow state change
Hi Florian,
Quoted 29 lines · Florian Westphal
We need to wakeup readers blocking for POLLIN, as no more data will arrive they will never be awoken otherwise.
This is needed at least until proper MPTCP-Level fin/reset signalling gets added.
Signed-off-by Florian Westphal <fw(a)strlen.de>---
net/mptcp/subflow.c| 23 +++++++++++++++++++++++1 file changed, 23 insertions(+)net/mptcp/subflow.c +10
-0@@ -722,6 +722,28 @@ static struct mptcp_subflow_context *subflow_create_ctx(struct sock *sk,return ctx; } +static void __subflow_state_change (struct sock *sk) +{ +struct socket_wq *wq; + +rcu_read_lock (); + wq =rcu_dereference (sk->sk_wq); +if (skwq_has_sleeper (wq)) +wake_up_interruptible_all (&wq->wait); +rcu_read_unlock (); +}
net/mptcp/subflow.c static struct mptcp_subflow_context *subflow_create_ctx(struct sock *sk, · line 744
Quoted 10 lines
net/mptcp/subflow.c +10
-0+ +static void subflow_state_change (struct sock *sk) +{ +struct mptcp_subflow_context *subflow =mptcp_subflow_ctx (sk); +struct sock *parent = subflow->conn; + +__subflow_state_change (sk); + +if (parent) +__subflow_state_change (parent);
In the v1 patch I asked about the missing sk_wake_async() for the parent socket - you proposed calling parent->sk_data_ready(parent) here. Can you confirm which is intended?
https://lists.01.org/hyperkitty/list/mptcp(a)lists.01.org/message/EMGGJL4WII7NYOVGY7VM7DAVYDGXSVJO/
Thanks,
Mat
Quoted 11 lines
net/mptcp/subflow.c +3
-0+} +static int subflow_ulp_init (struct sock *sk) {struct inet_connection_sock *icsk =inet_csk (sk);@@ -750,6 +772,7 @@ static int subflow_ulp_init(struct sock *sk) ctx->tcp_sk_data_ready = sk->sk_data_ready; sk->sk_data_ready = subflow_data_ready; sk->sk_write_space = subflow_write_space; + sk->sk_state_change = subflow_state_change;out:
Quoted 8 lines
return err; }Signature · 4 lines
2.21.0 _______________________________________________ mptcp mailing list -- mptcp(a)lists.01.org To unsubscribe send an email to mptcp-leave(a)lists.01.org
--
Mat Martineau
Intel
Florian Westphal
[MPTCP] Re: [PATCH v2 2/4] subflow: wake parent mptcp socket on subflow state change
Quoted 46 lines · Mat Martineau
Quoted 29 lines · Florian Westphal
We need to wakeup readers blocking for POLLIN, as no more data will arrive they will never be awoken otherwise.
This is needed at least until proper MPTCP-Level fin/reset signalling gets added.
Signed-off-by Florian Westphal <fw(a)strlen.de>---
net/mptcp/subflow.c| 23 +++++++++++++++++++++++1 file changed, 23 insertions(+)net/mptcp/subflow.c +10
-0@@ -722,6 +722,28 @@ static struct mptcp_subflow_context *subflow_create_ctx(struct sock *sk,return ctx; } +static void __subflow_state_change (struct sock *sk) +{ +struct socket_wq *wq; + +rcu_read_lock (); + wq =rcu_dereference (sk->sk_wq); +if (skwq_has_sleeper (wq)) +wake_up_interruptible_all (&wq->wait); +rcu_read_unlock (); +}net/mptcp/subflow.c static struct mptcp_subflow_context *subflow_create_ctx(struct sock *sk, · line 744
Quoted 10 lines
net/mptcp/subflow.c +10
-0+ +static void subflow_state_change (struct sock *sk) +{ +struct mptcp_subflow_context *subflow =mptcp_subflow_ctx (sk); +struct sock *parent = subflow->conn; + +__subflow_state_change (sk); + +if (parent) +__subflow_state_change (parent);In the v1 patch I asked about the missing
sk_wake_async()for the parent socket - you proposed callingparent->sk_data_ready(parent)here. Can you confirm which is intended?
Quoted 1 line
https://lists.01.org/hyperkitty/list/mptcp(a)lists.01.org/message/EMGGJL4WII7NYOVGY7VM7DAVYDGXSVJO/
That comment refers to subflow_data_ready(). The patch here was not part of v1. An alternative would be to make sock_def_wakeup() non-static and use it but since the function is very small I don't think thats needed. We will also likely want to do something different once MPTCP is able to process mptcp-level reset/fin.
The proposed change of
if (mptcp_should_wake_parent(subflow))
parent->sk_data_ready(parent);
inside subflow_data_ready isn't needed anymore because after Paolos refactoring the existing parent->sk_data_ready(parent);
is already conditional on 'we have in-sequence data', so I dropped that part from the last patch.
Mat Martineau
[MPTCP] Re: [PATCH v2 2/4] subflow: wake parent mptcp socket on subflow state change
Quoted 57 lines · Florian Westphal
Quoted 46 lines · Mat Martineau
Quoted 29 lines · Florian Westphal
We need to wakeup readers blocking for POLLIN, as no more data will arrive they will never be awoken otherwise.
This is needed at least until proper MPTCP-Level fin/reset signalling gets added.
Signed-off-by Florian Westphal <fw(a)strlen.de>---
net/mptcp/subflow.c| 23 +++++++++++++++++++++++1 file changed, 23 insertions(+)net/mptcp/subflow.c +10
-0@@ -722,6 +722,28 @@ static struct mptcp_subflow_context *subflow_create_ctx(struct sock *sk,return ctx; } +static void __subflow_state_change (struct sock *sk) +{ +struct socket_wq *wq; + +rcu_read_lock (); + wq =rcu_dereference (sk->sk_wq); +if (skwq_has_sleeper (wq)) +wake_up_interruptible_all (&wq->wait); +rcu_read_unlock (); +}net/mptcp/subflow.c static struct mptcp_subflow_context *subflow_create_ctx(struct sock *sk, · line 744
Quoted 10 lines
net/mptcp/subflow.c +10
-0+ +static void subflow_state_change (struct sock *sk) +{ +struct mptcp_subflow_context *subflow =mptcp_subflow_ctx (sk); +struct sock *parent = subflow->conn; + +__subflow_state_change (sk); + +if (parent) +__subflow_state_change (parent);In the v1 patch I asked about the missing
sk_wake_async()for the parent socket - you proposed callingparent->sk_data_ready(parent)here. Can you confirm which is intended?Quoted 1 line
https://lists.01.org/hyperkitty/list/mptcp(a)lists.01.org/message/EMGGJL4WII7NYOVGY7VM7DAVYDGXSVJO/
That comment refers to
subflow_data_ready(). The patch here was not part of v1. An alternative would be to makesock_def_wakeup()non-static and use it but since the function is very small I don't think thats needed. We will also likely want to do something different once MPTCP is able to process mptcp-level reset/fin.The proposed change of
Quoted 9 lines
if (
mptcp_should_wake_parent(subflow))parent->sk_data_ready(parent);inside
subflow_data_readyisn't needed anymore because after Paolos refactoring the existingparent->sk_data_ready(parent);is already conditional on 'we have in-sequence data', so I dropped that part from the last patch.
Yeah, I misread the new patch. Thanks for explaining. I do agree that it makes more sense to create __subflow_state_change() than it does to export sock_def_wakeup().
--
Mat Martineau
Intel
Florian Westphal
PATCH v2 3/4 net: tcp: add __tcp_poll helper
In reply to Florian WestphalMPTCP will need to be able to call tcp_poll to check if the tcp socket is available for reading/writing, but without adding the task to the waitlist.
This splits tcp_poll into two functions so __tcp_poll can be called by mptcp in followup patch.
--- include/net/tcp.h | 1 + net/ipv4/tcp.c | 12 ++++++++---- 2 files changed, 9 insertions(+), 4 deletions(-)
include/net/tcp.h +1 -0
@@ -396,6 +396,7 @@ void tcp_init_sock(struct sock *sk);void tcp_init_transfer (struct sock *sk,int bpf_op);__poll_t tcp_poll (struct file *file,struct socket *sock,struct poll_table_struct *wait); +__poll_t __tcp_poll (struct sock *sk);int tcp_getsockopt (struct sock *sk,int level,int optname,char __user *optval,int __user *optlen); int tcp_setsockopt(struct sock *sk, int level, int optname,
net/ipv4/tcp.c +8 -4
@@ -490,15 +490,12 @@ static inline bool tcp_stream_is_readable(const struct tcp_sock *tp,* take care of normal races (between the test and the event) and we don't * go look at any of the socket buffers directly. */ -__poll_t tcp_poll (struct file *file,struct socket *sock,poll_table *wait) +__poll_t __tcp_poll (struct sock *sk) {__poll_t mask; -struct sock *sk = sock->sk;const struct tcp_sock *tp =tcp_sk (sk);int state; -sock_poll_wait (file, sock, wait); - state =inet_sk_state_load (sk);if (state ==TCP_LISTEN )return inet_csk_listen_poll (sk);
@@ -590,6 +587,13 @@ __poll_t tcp_poll(struct file *file, struct socket *sock, poll_table *wait)return mask; } + +__poll_t tcp_poll (struct file *file,struct socket *sock,poll_table *wait) +{ +sock_poll_wait (file, sock, wait); + +return __tcp_poll (sock->sk); +}EXPORT_SYMBOL (tcp_poll);int tcp_ioctl (struct sock *sk,int cmd,unsigned long arg)
Signature · 1 line
2.21.0
Florian Westphal
PATCH v2 4/4 mptcp: switch mptcp_poll to mptcp socket wait queue
In reply to Florian Westphaltcp_poll places current task on a wait queue. This can cause us to block forever -- some subflows might never receive any data because they're used as a backup/fallback path.
This makes mptcp_poll wait on the mptcp socket wait queue instead. __tcp_poll is used instead to tcp_poll so current task won't wait again after mptcp socket has been woken from the subflow layer.
--- net/mptcp/protocol.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +3 -1
@@ -1238,11 +1238,13 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,return ret; } +sock_poll_wait (file, sock, wait); +mptcp_for_each_subflow (msk, subflow) {struct socket *tcp_sock; tcp_sock =mptcp_subflow_tcp_socket (subflow); - ret |=tcp_poll (file, tcp_sock, wait); + ret |=__tcp_poll (tcp_sock->sk); }release_sock (sk);
Signature · 1 line
2.21.0
Paolo Abeni
Quoted 32 lines · Florian Westphal
No major changes, except the rebase on current export branch. Selftest still passes.
This series is needed for sendmsg refactor, without it we can't make mptcp sendmsg wait on the mptcp socket wait queue because we don't get the needed wakeup events from the subflow layer.
The following changes since commit
85c93de446d3fd90e47605ba8ff161bef7558f09:mptcp: Implement basic path manager (2019-10-23 01:31:04 +0000)
are available in the Git repository at:
git://git.breakpoint.cc/fw/mptcp-next.gitmptcp_poll_07for you to fetch changes up to
864dd47df1293e4b8bfc70be0d459286b6dddc3a:mptcp: switch
mptcp_pollto mptcp socket wait queue (2019-10-23 15:20:28 +0200)----------------------------------------------------------------
Florian Westphal (4):
mptcp: add and usesubflow_write_space
subflow: wake parent mptcp socket on subflow state change
net: tcp: add__tcp_pollhelper
mptcp: switchmptcp_pollto mptcp socket wait queueinclude/net/tcp.h | 1 + net/ipv4/tcp.c | 12 ++++++++---- net/mptcp/protocol.c | 4 +++- net/mptcp/subflow.c | 34 ++++++++++++++++++++++++++++++++++ 4 files changed, 46 insertions(+), 5 deletions(-)
Quoted 3 lines
_______________________________________________
mptcp mailing list -- mptcp(a)lists.01.org
To unsubscribe send an email to mptcp-leave(a)lists.01.org
Series LGTM
I think we could include patch 3/4 in the "first chunk"/preparatory patches - e.g. apply just after "tcp, ulp: Add clone operation to tcp_ulp_ops" - and squash 4/4 in "mptcp: add mptcp_poll"
WDYT?
Thanks,
Paolo
Florian Westphal
Quoted 37 lines · Paolo Abeni
Quoted 32 lines · Florian Westphal
No major changes, except the rebase on current export branch. Selftest still passes.
This series is needed for sendmsg refactor, without it we can't make mptcp sendmsg wait on the mptcp socket wait queue because we don't get the needed wakeup events from the subflow layer.
The following changes since commit
85c93de446d3fd90e47605ba8ff161bef7558f09:mptcp: Implement basic path manager (2019-10-23 01:31:04 +0000)
are available in the Git repository at:
git://git.breakpoint.cc/fw/mptcp-next.gitmptcp_poll_07for you to fetch changes up to
864dd47df1293e4b8bfc70be0d459286b6dddc3a:mptcp: switch
mptcp_pollto mptcp socket wait queue (2019-10-23 15:20:28 +0200)----------------------------------------------------------------
Florian Westphal (4):
mptcp: add and usesubflow_write_space
subflow: wake parent mptcp socket on subflow state change
net: tcp: add__tcp_pollhelper
mptcp: switchmptcp_pollto mptcp socket wait queueinclude/net/tcp.h | 1 + net/ipv4/tcp.c | 12 ++++++++---- net/mptcp/protocol.c | 4 +++- net/mptcp/subflow.c | 34 ++++++++++++++++++++++++++++++++++ 4 files changed, 46 insertions(+), 5 deletions(-)Quoted 3 lines
_______________________________________________
mptcp mailing list -- mptcp(a)lists.01.org
To unsubscribe send an email to mptcp-leave(a)lists.01.org
Quoted 7 lines
Series LGTM
I think we could include patch 3/4 in the "first chunk"/preparatory patches - e.g. apply just after "tcp, ulp: Add clone operation to
tcp_ulp_ops" - and squash 4/4 in "mptcp: addmptcp_poll"WDYT?
Seems good, yes.
Matthieu Baerts
Hi Paolo, Florian,
Quoted 40 lines · Florian Westphal
Quoted 39 lines · Paolo Abeni
Quoted 32 lines · Florian Westphal
No major changes, except the rebase on current export branch. Selftest still passes.
This series is needed for sendmsg refactor, without it we can't make mptcp sendmsg wait on the mptcp socket wait queue because we don't get the needed wakeup events from the subflow layer.
The following changes since commit
85c93de446d3fd90e47605ba8ff161bef7558f09:mptcp: Implement basic path manager (2019-10-23 01:31:04 +0000)
are available in the Git repository at:
git://git.breakpoint.cc/fw/mptcp-next.gitmptcp_poll_07for you to fetch changes up to
864dd47df1293e4b8bfc70be0d459286b6dddc3a:mptcp: switch
mptcp_pollto mptcp socket wait queue (2019-10-23 15:20:28 +0200)----------------------------------------------------------------
Florian Westphal (4):
mptcp: add and usesubflow_write_space
subflow: wake parent mptcp socket on subflow state change
net: tcp: add__tcp_pollhelper
mptcp: switchmptcp_pollto mptcp socket wait queueinclude/net/tcp.h | 1 + net/ipv4/tcp.c | 12 ++++++++---- net/mptcp/protocol.c | 4 +++- net/mptcp/subflow.c | 34 ++++++++++++++++++++++++++++++++++ 4 files changed, 46 insertions(+), 5 deletions(-)Quoted 3 lines
_______________________________________________
mptcp mailing list -- mptcp(a)lists.01.org
To unsubscribe send an email to mptcp-leave(a)lists.01.orgSeries LGTM
Quoted 7 lines
Quoted 5 lines
I think we could include patch 3/4 in the "first chunk"/preparatory patches - e.g. apply just after "tcp, ulp: Add clone operation to
tcp_ulp_ops" - and squash 4/4 in "mptcp: addmptcp_poll"WDYT?
Seems good, yes.
Where can I apply patch 1/4 and 2/4? At the end?
Cheers,
Matt
Signature · 5 lines
Matthieu Baerts | R&D Engineer matthieu.baerts(a)tessares.net Tessares SA | Hybrid Access Solutions www.tessares.net 1 Avenue Jean Monnet, 1348 Louvain-la-Neuve, Belgium
Florian Westphal
Quoted 10 lines · Matthieu Baerts
Quoted 8 lines
Quoted 6 lines
I think we could include patch 3/4 in the "first chunk"/preparatory patches - e.g. apply just after "tcp, ulp: Add clone operation to
tcp_ulp_ops" - and squash 4/4 in "mptcp: addmptcp_poll"WDYT?
Seems good, yes.
Where can I apply patch 1/4 and 2/4? At the end?
1 could probably be squashed with "mptcp: Implement MPTCP receive path", given that patch adds sk_data_ready for subflows.
Patch 2 I'd add at the end, we might be able to revert it eventually.
Matthieu Baerts
Hi Florian,
Quoted 8 lines · Florian Westphal
Quoted 7 lines · Matthieu Baerts
Quoted 7 lines
Quoted 6 lines
I think we could include patch 3/4 in the "first chunk"/preparatory patches - e.g. apply just after "tcp, ulp: Add clone operation to
tcp_ulp_ops" - and squash 4/4 in "mptcp: addmptcp_poll"WDYT?
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Seems good, yes.
Where can I apply patch 1/4 and 2/4? At the end?
1 could probably be squashed with "mptcp: Implement MPTCP receive path", given that patch adds
sk_data_readyfor subflows.Patch 2 I'd add at the end, we might be able to revert it eventually.
Thank you for the patches and the quick reply!
And thank you Paolo and Mat for the reviews!
- af383a1b6937: "squashed" patch 1/4 in "mptcp: Implement MPTCP receive path"
- 8e678a61f7b7: "Signed-off-by" + "Co-developed-by"
- 77cf55a4b1f1: conflict in t/mptcp-Add-handling-of-outgoing-MP_JOIN-requests
- 789fc73cc101..c9ae282e1e87: result
- f0d0353bc148: patch 2 added at the end
- c9ae282e1e87..590467e30328: result
- 67590165a990: patch 3 added between "tcp, ulp: Add clone operation to tcp_ulp_ops" and "mptcp: Add MPTCP to skb extensions"
- 590467e30328..555678057b0c: result
- b52d7b74d906: "squashed" patch 4/4 in "mptcp: add mptcp_poll"
- 555678057b0c..460e610b3ca4: result
Tests are in progress, verifying that each commit can still compile.
Cheers,
Matt
Signature · 5 lines
Matthieu Baerts | R&D Engineer matthieu.baerts(a)tessares.net Tessares SA | Hybrid Access Solutions www.tessares.net 1 Avenue Jean Monnet, 1348 Louvain-la-Neuve, Belgium
Mat Martineau
Quoted 26 lines · Florian Westphal
No major changes, except the rebase on current export branch. Selftest still passes.
This series is needed for sendmsg refactor, without it we can't make mptcp sendmsg wait on the mptcp socket wait queue because we don't get the needed wakeup events from the subflow layer.
The following changes since commit
85c93de446d3fd90e47605ba8ff161bef7558f09:mptcp: Implement basic path manager (2019-10-23 01:31:04 +0000)
are available in the Git repository at:
git://git.breakpoint.cc/fw/mptcp-next.gitmptcp_poll_07for you to fetch changes up to
864dd47df1293e4b8bfc70be0d459286b6dddc3a:mptcp: switch
mptcp_pollto mptcp socket wait queue (2019-10-23 15:20:28 +0200)----------------------------------------------------------------
Florian Westphal (4):
mptcp: add and usesubflow_write_space
subflow: wake parent mptcp socket on subflow state change
net: tcp: add__tcp_pollhelper
mptcp: switchmptcp_pollto mptcp socket wait queue
Quoted 6 lines
include/net/tcp.h | 1 + net/ipv4/tcp.c | 12 ++++++++---- net/mptcp/protocol.c | 4 +++- net/mptcp/subflow.c | 34 ++++++++++++++++++++++++++++++++++ 4 files changed, 46 insertions(+), 5 deletions(-)
This series looks ready to merge. Thanks, Florian!
--
Mat Martineau
Intel