mptcp_poll rework preview
Subsystems (3)
Florian Westphal
cover mptcp_poll rework preview
This patch series is in need of a bit more work, f.e. there is too much copypastry in patch 3.
However, it is functional as-is so I am sharing it to get earlier reviews.
Problem with current mptcp_poll() is that we can't block the task on each
subflow: subflow we're polling on might be idle while another one has data ready for processing.
include/net/tcp.h | 1 net/ipv4/tcp.c | 12 +++- net/mptcp/protocol.c | 131 ++++++++++++++++++++++++++++++++++++++++++++++----- net/mptcp/protocol.h | 1 net/mptcp/subflow.c | 24 +++++++-- 5 files changed, 150 insertions(+), 19 deletions(-)
Florian Westphal
RFC PATCH 1/6 protocol: add discard_one_skb helper
This can be re-used in followup patch.
--- net/mptcp/protocol.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-)
net/mptcp/protocol.c +12 -10
@@ -384,6 +384,15 @@ static enum mapping_status mptcp_get_mapping(struct sock *ssk)return ret; } +static bool mptcp_discard_one_skb (struct sock *sk) +{ +read_descriptor_t rd = { + .count =SIZE_MAX , + }; + +return tcp_read_sock (sk, &rd, mptcp_flush_actor) >=0 ; +} +static void mptcp_wait_data (struct sock *sk,long *timeo) {DEFINE_WAIT_FUNC (wait, woken_wake_function);
@@ -471,16 +480,9 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,if (!subflow->map_valid) {pr_debug ("Mapping missing, trying next skb" ); - arg.msg =NULL ; - desc.count =SIZE_MAX ; - - bytes_read =tcp_read_sock (ssk, &desc, - mptcp_flush_actor); - -if (bytes_read <0 ) -break ; - -continue ; +if (mptcp_discard_one_skb (ssk)) +continue ; +break ; } }else if (status ==MAPPING_EMPTY ) { goto wait_for_data;
Signature · 1 line
2.21.0
Florian Westphal
RFC PATCH 2/6 mptcp: add and use subflow_write_space
In reply to Florian WestphalThis will be used to unblock tasks waiting for POLLOUT event once mptcp_poll is altered to not wait for each subflow anymore.
--- net/mptcp/subflow.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
net/mptcp/subflow.c +11 -0
@@ -318,6 +318,16 @@ static void subflow_data_ready(struct sock *sk) } } +static void subflow_write_space (struct sock *sk) +{ +struct subflow_context *subflow =subflow_ctx (sk); +struct sock *parent = subflow->conn; + +sk_stream_write_space (sk); +if (parent) + parent->sk_write_space (parent); +} +int subflow_connect (struct sock *sk,struct sockaddr_in *local,struct sockaddr_in *remote,u8 remote_id) {
@@ -436,6 +446,7 @@ static int subflow_ulp_init(struct sock *sk) ctx->tcp_sk_data_ready = sk->sk_data_ready; ctx->use_checksum =0 ; sk->sk_data_ready = subflow_data_ready; + sk->sk_write_space = subflow_write_space; out:return err; }
Signature · 1 line
2.21.0
Florian Westphal
RFC PATCH 3/6 mptcp: add mptcp_should_wake_parent
In reply to Florian WestphalThe function is never called at this time, because mptcp_poll() places the task on the subflow poll list, instead of the mptcp socket one.
Will be changed in the followup patch.
Note that this adds quite some code duplication, more code sharing with recv path is needed here.
--- net/mptcp/protocol.c | 99 ++++++++++++++++++++++++++++++++++++++++++++ net/mptcp/protocol.h | 1 + net/mptcp/subflow.c | 13 ++++-- 3 files changed, 109 insertions(+), 4 deletions(-)
net/mptcp/protocol.c +99 -0
@@ -393,6 +393,105 @@ static bool mptcp_discard_one_skb(struct sock *sk)return tcp_read_sock (sk, &rd, mptcp_flush_actor) >=0 ; } +bool mptcp_must_wake_parent (struct subflow_context *subflow) +{ +struct socket *socket =mptcp_subflow_tcp_socket (subflow); +struct sock *sk = subflow->conn; +struct sock *ssk = socket->sk; +struct mptcp_sock *msk; +int state; + + state =inet_sk_state_load (sk); + msk =mptcp_sk (sk); +if (state !=TCP_ESTABLISHED ) +return true; + +sock_owned_by_me ((const struct sock *)ssk); + +for (;;) { +enum mapping_status status; +size_t discard_len =0 ; +u64 ack_seq, old_ack; +int bytes_read; +u32 ssn; + + state =inet_sk_state_load (ssk); + status =mptcp_get_mapping (ssk); +switch (status) { +case MAPPING_ADDED : +break ; +case MAPPING_MISSING : +if (!subflow->map_valid) { +if (mptcp_discard_one_skb (ssk)) +continue ; + +return true; + } +/* fall through */ +case MAPPING_EMPTY : +if (state ==TCP_ESTABLISHED ) +return false; + +/* The fact that one subflow is closing should NOT result +* in a wakeup. However, we need this for now so EOF can +* be signalled. We should instead check if the logical +* mptcp connection state has changed. +*/ +return true; +case MAPPING_DATA_FIN : +return true; + } + + ssn =tcp_sk (ssk)->copied_seq - subflow->ssn_offset; + old_ack =READ_ONCE (msk->ack_seq); + +if (unlikely (before (ssn, subflow->map_subflow_seq))) { +/* Mapping covers data later in the subflow stream */ + discard_len = subflow->map_subflow_seq - ssn; + }else if (unlikely (!before (ssn, (subflow->map_subflow_seq + + subflow->map_data_len)))) { +/* Mapping ends earlier in the subflow stream. +* Invalidate the mapping and try again. +*/ + subflow->map_valid =0 ; +continue ; + }else { + ack_seq =get_mapped_dsn (subflow); + +if (before64 (ack_seq, old_ack)) { +/* Mapping covers data already received, +* discard data in the current mapping +* and invalidate the map +*/ +u64 map_end_dsn = subflow->map_seq + + subflow->map_data_len; + discard_len =min (map_end_dsn - ack_seq, + old_ack - ack_seq); + } + } + +if (discard_len) { +struct mptcp_read_arg arg = { + .msg =NULL , + }; +read_descriptor_t desc = { + .count = discard_len, + .arg.data = &arg, + }; + + bytes_read =tcp_read_sock (ssk, &desc, + mptcp_read_actor); +if (bytes_read <0 ) +break ; +else if (bytes_read == discard_len) +continue ; + } +break ; + } + +return true; +} +static void mptcp_wait_data (struct sock *sk,long *timeo) {DEFINE_WAIT_FUNC (wait, woken_wake_function);
net/mptcp/protocol.h +1 -0
@@ -228,6 +228,7 @@ void mptcp_get_options(const struct sk_buff *skb,void mptcp_finish_connect (struct sock *sk,int mp_capable);void mptcp_finish_join (struct sock *sk); +bool mptcp_must_wake_parent (struct subflow_context *ctx);int mptcp_token_new_request (struct request_sock *req);void mptcp_token_destroy_request (u32 token);
net/mptcp/subflow.c +9 -4
@@ -304,18 +304,23 @@ static void subflow_data_ready(struct sock *sk)struct subflow_context *subflow =subflow_ctx (sk);struct sock *parent = subflow->conn; -pr_debug ("sk=%p" , sk); subflow->tcp_sk_data_ready (sk);if (parent) { -pr_debug ("parent=%p" , parent); +struct socket_wq *wq;smp_mb__before_atomic ();set_bit (MPTCP_DATA_READY , &mptcp_sk (parent)->flags);smp_mb__after_atomic (); - parent->sk_data_ready (parent); - } +rcu_read_lock (); + wq =rcu_dereference (parent->sk_wq); +if (skwq_has_sleeper (wq) && +mptcp_must_wake_parent (subflow)) { +wake_up_interruptible_all (&wq->wait); + } +rcu_read_unlock (); + } }static void subflow_write_space (struct sock *sk)
Signature · 1 line
2.21.0
Florian Westphal
RFC PATCH 4/6 mptcp: enable must_wake_parent helper
In reply to Florian WestphalIf task calls poll(), wait on mptcp wait queue before the subflow wait lists.
Followup patch will remove subflow wait list placement entirely.
--- net/mptcp/protocol.c | 8 ++++++++ 1 file changed, 8 insertions(+)
net/mptcp/protocol.c +8 -0
@@ -406,6 +406,9 @@ bool mptcp_must_wake_parent(struct subflow_context *subflow)if (state !=TCP_ESTABLISHED )return true; +if (WARN_ON_ONCE (!tcp_sk (ssk)->is_mptcp)) +return true; +sock_owned_by_me ((const struct sock *)ssk);for (;;) {
@@ -458,6 +461,8 @@ bool mptcp_must_wake_parent(struct subflow_context *subflow) }else { ack_seq =get_mapped_dsn (subflow); +WARN_ON_ONCE (!subflow->map_valid); +if (before64 (ack_seq, old_ack)) {/* Mapping covers data already received, * discard data in the current mapping
@@ -1170,6 +1175,9 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,return ret; } +sock_owned_by_me ((const struct sock *)msk); +sock_poll_wait (file, sock, wait); +mptcp_for_each_subflow (msk, subflow) {struct socket *tcp_sock;
Signature · 1 line
2.21.0
Florian Westphal
RFC PATCH 5/6 net: tcp: add __tcp_poll helper
In reply to Florian WestphalMPTCP will need to be able to call tcp_poll to check if the tcp socket is available for reading/writing, but without adding the task to the waitlist.
This splits tcp_poll into two functions so __tcp_poll can be called by mptcp in followup patch.
--- include/net/tcp.h | 1 + net/ipv4/tcp.c | 12 ++++++++---- 2 files changed, 9 insertions(+), 4 deletions(-)
include/net/tcp.h +1 -0
@@ -396,6 +396,7 @@ void tcp_init_sock(struct sock *sk);void tcp_init_transfer (struct sock *sk,int bpf_op);__poll_t tcp_poll (struct file *file,struct socket *sock,struct poll_table_struct *wait); +__poll_t __tcp_poll (struct sock *sk);int tcp_getsockopt (struct sock *sk,int level,int optname,char __user *optval,int __user *optlen); int tcp_setsockopt(struct sock *sk, int level, int optname,
net/ipv4/tcp.c +8 -4
@@ -490,15 +490,12 @@ static inline bool tcp_stream_is_readable(const struct tcp_sock *tp,* take care of normal races (between the test and the event) and we don't * go look at any of the socket buffers directly. */ -__poll_t tcp_poll (struct file *file,struct socket *sock,poll_table *wait) +__poll_t __tcp_poll (struct sock *sk) {__poll_t mask; -struct sock *sk = sock->sk;const struct tcp_sock *tp =tcp_sk (sk);int state; -sock_poll_wait (file, sock, wait); - state =inet_sk_state_load (sk);if (state ==TCP_LISTEN )return inet_csk_listen_poll (sk);
@@ -590,6 +587,13 @@ __poll_t tcp_poll(struct file *file, struct socket *sock, poll_table *wait)return mask; } + +__poll_t tcp_poll (struct file *file,struct socket *sock,poll_table *wait) +{ +sock_poll_wait (file, sock, wait); + +return __tcp_poll (sock->sk); +}EXPORT_SYMBOL (tcp_poll);int tcp_ioctl (struct sock *sk,int cmd,unsigned long arg)
Signature · 1 line
2.21.0
Florian Westphal
RFC PATCH 6/6 mptcp: use __tcp_poll instead of tcp_poll
In reply to Florian Westphaltcp_poll places current task on a wait queue. This can cause us to block forever -- some subflows might be backup/failover and never receive any data.
Use __tcp_poll instead to avoid blocking current task again after parent mptcp socket has been woken up from the subflow layer.
--- net/mptcp/protocol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/mptcp/protocol.c +1 -1
@@ -1182,7 +1182,7 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,struct socket *tcp_sock; tcp_sock =mptcp_subflow_tcp_socket (subflow); - ret |=tcp_poll (file, tcp_sock, wait); + ret |=__tcp_poll (tcp_sock->sk); }release_sock (sk);
Signature · 1 line
2.21.0
Paolo Abeni
Re: [MPTCP] [PATCH RFC 6/6] mptcp: use __tcp_poll instead of tcp_poll
Quoted 17 lines · Florian Westphal
tcp_pollplaces current task on a wait queue. This can cause us to block forever -- some subflows might be backup/failover and never receive any data.Use
__tcp_pollinstead to avoid blocking current task again after parent mptcp socket has been woken up from the subflow layer.Signed-off-by Florian Westphal <fw(a)strlen.de>--- net/mptcp/protocol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/mptcp/protocol.c +0
-0
net/mptcp/protocol.c static __poll_t mptcp_poll(struct file *file, struct socket *sock, · line 1187
Quoted 7 lines
net/mptcp/protocol.c +1
-1@@ -1182,7 +1182,7 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,struct socket *tcp_sock; tcp_sock =mptcp_subflow_tcp_socket (subflow); - ret |=tcp_poll (file, tcp_sock, wait); + ret |=__tcp_poll (tcp_sock->sk); }release_sock (sk);
This allows mptcp_poll working even without a 'struct socket'. we still relies on such struct presence for:
kernel_acceptkernel_sock_shutdownkernel_setsockoptkernel_getsockoptkernel_bindkernel_connect
I'm wondering if we should try to get rid completely of any depenency on 'struct socket'? That will fix the current issue with MP_JOIN, will reduce the memory usage and likely will improve accept time.
A quick look makes me think that the most problematic call to be addressed without using any 'struct socket' are kernel_bind() and kernel_connect(), perhaps we can use a transient 'struct socket' just for subflow_connect() ? WDYT?
Thanks,
Paolo
Florian Westphal
Re: [MPTCP] [PATCH RFC 6/6] mptcp: use __tcp_poll instead of tcp_poll
Quoted 8 lines · Paolo Abeni
net/mptcp/protocol.c static __poll_t mptcp_poll(struct file *file, struct socket *sock, · line 1187
Quoted 5 lines
net/mptcp/protocol.c +1
-1tcp_sock =mptcp_subflow_tcp_socket (subflow); - ret |=tcp_poll (file, tcp_sock, wait); + ret |=__tcp_poll (tcp_sock->sk); }release_sock (sk);This allows
mptcp_pollworking even without a 'struct socket'. we still relies on such struct presence for:
Thats what I though as well, but tcp_poll assumes sk->sk_socket points to the userspace-facing socket structure.
Quoted 10 lines
kernel_acceptkernel_sock_shutdownkernel_setsockoptkernel_getsockoptkernel_bindkernel_connectI'm wondering if we should try to get rid completely of any depenency on 'struct socket'? That will fix the current issue with
MP_JOIN, will reduce the memory usage and likely will improve accept time.
It will require careful audit of the stack, afaics Tcp assumes that for anythig incoming tcp sk might have been detached from usersapce already (i.e., application called close()). But for anything coming from syscalls that doesn't seem to be the case.
I started to hack up an MP_JOIN patch that uses a per-sk workqueue to defer incoming joins so that a socket can be allocated for it.
I don't like it, but it seems easier to go with that route.
Paolo Abeni
Hi,
Quoted 8 lines · Florian Westphal
This patch series is in need of a bit more work, f.e. there is too much copypastry in patch 3.
However, it is functional as-is so I am sharing it to get earlier reviews.
Problem with current
mptcp_poll()is that we can't block the task on each
subflow: subflow we're polling on might be idle while another one has data ready for processing.
If I read the series correctly, patch 3 is not strictly needed for the above goal: AFAICS it should help to avoid empty/blocking recvmsg() after successful poll(), right?
I think we should rework a bit recvmsg() before going on with this - see "mptcp: partial recvmsg() refactor". Perhaps even something more drastic e.g. subflow could try move skbs from ssk->sk_receive_queue to a per msk queue, ordered by mptcp seq in subflow_data_ready(). WDYT?
Cheers,
Paolo
Florian Westphal
Quoted 6 lines · Paolo Abeni
Quoted 5 lines · Florian Westphal
This patch series is in need of a bit more work, f.e. there is too much copypastry in patch 3.
However, it is functional as-is so I am sharing it to get earlier reviews.
Quoted 7 lines
Quoted 3 lines
Problem with current
mptcp_poll()is that we can't block the task on each
subflow: subflow we're polling on might be idle while another one has data ready for processing.If I read the series correctly, patch 3 is not strictly needed for the above goal: AFAICS it should help to avoid empty/blocking
recvmsg()after successfulpoll(), right?
Thats right. EPOLLIN means next recv won't block, which isn't the case at this time because we do mptcp processing in recv only.
Quoted 2 lines
I think we should rework a bit
recvmsg()before going on with this - see "mptcp: partialrecvmsg()refactor".
Agree.
Quoted 3 lines
Perhaps even something more
drastic e.g. subflow could try move skbs fromssk->sk_receive_queueto a per msk queue, ordered by mptcp seq insubflow_data_ready(). WDYT?
Yes, that might be a good idea/needed.
We shouldn't depend on regular recv calls for mptcp processing.