Skip to contentSkip to conversation

mptcp_poll rework preview

RFC PATCH In review Florian Westphal 2 participants 11 messages Sep 9, 2019 – Sep 17, 2019

Subsystems (3)
lore
Share & tools
lore

Florian Westphal

author
#0 lore raw

cover mptcp_poll rework preview

This patch series is in need of a bit more work, f.e. there is too much copypastry in patch 3.

However, it is functional as-is so I am sharing it to get earlier reviews.

Problem with current mptcp_poll() is that we can't block the task on each
subflow: subflow we're polling on might be idle while another one has data ready for processing.

 include/net/tcp.h    |    1 
 net/ipv4/tcp.c       |   12 +++-
 net/mptcp/protocol.c |  131 ++++++++++++++++++++++++++++++++++++++++++++++-----
 net/mptcp/protocol.h |    1 
 net/mptcp/subflow.c  |   24 +++++++--
 5 files changed, 150 insertions(+), 19 deletions(-)
author
#1 lore raw

RFC PATCH 1/6 protocol: add discard_one_skb helper

This can be re-used in followup patch.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)
net/mptcp/protocol.c +12 -10
index 266a1028c9fd..3eecd9ae0c77 100644
@@ -384,6 +384,15 @@ static enum mapping_status mptcp_get_mapping(struct sock *ssk)
 	return ret;
 }
 
+static bool mptcp_discard_one_skb(struct sock *sk)
+{
+	read_descriptor_t rd = {
+		.count = SIZE_MAX,
+	};
+
+	return tcp_read_sock(sk, &rd, mptcp_flush_actor) >= 0;
+}
+
 static void mptcp_wait_data(struct sock *sk, long *timeo)
 {
 	DEFINE_WAIT_FUNC(wait, woken_wake_function);
@@ -471,16 +480,9 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 			if (!subflow->map_valid) {
 				pr_debug("Mapping missing, trying next skb");
 
-				arg.msg = NULL;
-				desc.count = SIZE_MAX;
-
-				bytes_read = tcp_read_sock(ssk, &desc,
-							   mptcp_flush_actor);
-
-				if (bytes_read < 0)
-					break;
-
-				continue;
+				if (mptcp_discard_one_skb(ssk))
+					continue;
+				break;
 			}
 		} else if (status == MAPPING_EMPTY) {
 			goto wait_for_data;
Signature · 1 line
2.21.0
author
#2 lore raw

RFC PATCH 2/6 mptcp: add and use subflow_write_space

In reply to Florian Westphal

This will be used to unblock tasks waiting for POLLOUT event once mptcp_poll is altered to not wait for each subflow anymore.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/subflow.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
net/mptcp/subflow.c +11 -0
index 468f9b2aeb30..5635e4c73e15 100644
@@ -318,6 +318,16 @@ static void subflow_data_ready(struct sock *sk)
 	}
 }
 
+static void subflow_write_space(struct sock *sk)
+{
+	struct subflow_context *subflow = subflow_ctx(sk);
+	struct sock *parent = subflow->conn;
+
+	sk_stream_write_space(sk);
+	if (parent)
+		parent->sk_write_space(parent);
+}
+
 int subflow_connect(struct sock *sk, struct sockaddr_in *local,
 		    struct sockaddr_in *remote, u8 remote_id)
 {
@@ -436,6 +446,7 @@ static int subflow_ulp_init(struct sock *sk)
 	ctx->tcp_sk_data_ready = sk->sk_data_ready;
 	ctx->use_checksum = 0;
 	sk->sk_data_ready = subflow_data_ready;
+	sk->sk_write_space = subflow_write_space;
 out:
 	return err;
 }
Signature · 1 line
2.21.0
author
#3 lore raw

RFC PATCH 3/6 mptcp: add mptcp_should_wake_parent

In reply to Florian Westphal

The function is never called at this time, because mptcp_poll() places the task on the subflow poll list, instead of the mptcp socket one.

Will be changed in the followup patch.

Note that this adds quite some code duplication, more code sharing with recv path is needed here.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 99 ++++++++++++++++++++++++++++++++++++++++++++
 net/mptcp/protocol.h |  1 +
 net/mptcp/subflow.c  | 13 ++++--
 3 files changed, 109 insertions(+), 4 deletions(-)
net/mptcp/protocol.c +99 -0
index 3eecd9ae0c77..071260f4f0b8 100644
@@ -393,6 +393,105 @@ static bool mptcp_discard_one_skb(struct sock *sk)
 	return tcp_read_sock(sk, &rd, mptcp_flush_actor) >= 0;
 }
 
+bool mptcp_must_wake_parent(struct subflow_context *subflow)
+{
+	struct socket *socket = mptcp_subflow_tcp_socket(subflow);
+	struct sock *sk = subflow->conn;
+	struct sock *ssk = socket->sk;
+	struct mptcp_sock *msk;
+	int state;
+
+	state = inet_sk_state_load(sk);
+	msk = mptcp_sk(sk);
+	if (state != TCP_ESTABLISHED)
+		return true;
+
+	sock_owned_by_me((const struct sock *)ssk);
+
+	for (;;) {
+		enum mapping_status status;
+		size_t discard_len = 0;
+		u64 ack_seq, old_ack;
+		int bytes_read;
+		u32 ssn;
+
+		state = inet_sk_state_load(ssk);
+		status = mptcp_get_mapping(ssk);
+		switch (status) {
+		case MAPPING_ADDED:
+			break;
+		case MAPPING_MISSING:
+			if (!subflow->map_valid) {
+				if (mptcp_discard_one_skb(ssk))
+					continue;
+
+				return true;
+			}
+			/* fall through */
+		case MAPPING_EMPTY:
+			if (state == TCP_ESTABLISHED)
+				return false;
+
+			/* The fact that one subflow is closing should NOT result
+			 * in a wakeup.  However, we need this for now so EOF can
+			 * be signalled.  We should instead check if the logical
+			 * mptcp connection state has changed.
+			 */
+			return true;
+		case MAPPING_DATA_FIN:
+			return true;
+		}
+
+		ssn = tcp_sk(ssk)->copied_seq - subflow->ssn_offset;
+		old_ack = READ_ONCE(msk->ack_seq);
+
+		if (unlikely(before(ssn, subflow->map_subflow_seq))) {
+			/* Mapping covers data later in the subflow stream */
+			discard_len = subflow->map_subflow_seq - ssn;
+		} else if (unlikely(!before(ssn, (subflow->map_subflow_seq +
+						  subflow->map_data_len)))) {
+			/* Mapping ends earlier in the subflow stream.
+			 * Invalidate the mapping and try again.
+			 */
+			subflow->map_valid = 0;
+			continue;
+		} else {
+			ack_seq = get_mapped_dsn(subflow);
+
+			if (before64(ack_seq, old_ack)) {
+				/* Mapping covers data already received,
+				 * discard data in the current mapping
+				 * and invalidate the map
+				 */
+				u64 map_end_dsn = subflow->map_seq +
+					subflow->map_data_len;
+				discard_len = min(map_end_dsn - ack_seq,
+						  old_ack - ack_seq);
+			}
+		}
+
+		if (discard_len) {
+			struct mptcp_read_arg arg = {
+				.msg = NULL,
+			};
+			read_descriptor_t desc = {
+				.count = discard_len,
+				.arg.data = &arg,
+			};
+
+			bytes_read = tcp_read_sock(ssk, &desc,
+						   mptcp_read_actor);
+			if (bytes_read < 0)
+				break;
+			else if (bytes_read == discard_len)
+				continue;
+		}
+		break;
+	}
+
+	return true;
+}
+
 static void mptcp_wait_data(struct sock *sk, long *timeo)
 {
 	DEFINE_WAIT_FUNC(wait, woken_wake_function);
net/mptcp/protocol.h +1 -0
index bd7b3b107fe0..58015d4e25be 100644
@@ -228,6 +228,7 @@ void mptcp_get_options(const struct sk_buff *skb,
 
 void mptcp_finish_connect(struct sock *sk, int mp_capable);
 void mptcp_finish_join(struct sock *sk);
+bool mptcp_must_wake_parent(struct subflow_context *ctx);
 
 int mptcp_token_new_request(struct request_sock *req);
 void mptcp_token_destroy_request(u32 token);
net/mptcp/subflow.c +9 -4
index 5635e4c73e15..99886d07c26d 100644
@@ -304,18 +304,23 @@ static void subflow_data_ready(struct sock *sk)
 	struct subflow_context *subflow = subflow_ctx(sk);
 	struct sock *parent = subflow->conn;
 
-	pr_debug("sk=%p", sk);
 	subflow->tcp_sk_data_ready(sk);
 
 	if (parent) {
-		pr_debug("parent=%p", parent);
+		struct socket_wq *wq;
 
 		smp_mb__before_atomic();
 		set_bit(MPTCP_DATA_READY, &mptcp_sk(parent)->flags);
 		smp_mb__after_atomic();
 
-		parent->sk_data_ready(parent);
-	}
+		rcu_read_lock();
+		wq = rcu_dereference(parent->sk_wq);
+		if (skwq_has_sleeper(wq) &&
+		    mptcp_must_wake_parent(subflow)) {
+			wake_up_interruptible_all(&wq->wait);
+		}
+		rcu_read_unlock();
+       }
 }
 
 static void subflow_write_space(struct sock *sk)
Signature · 1 line
2.21.0
author
#4 lore raw

RFC PATCH 4/6 mptcp: enable must_wake_parent helper

In reply to Florian Westphal

If task calls poll(), wait on mptcp wait queue before the subflow wait lists.

Followup patch will remove subflow wait list placement entirely.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 8 ++++++++
 1 file changed, 8 insertions(+)
net/mptcp/protocol.c +8 -0
index 071260f4f0b8..2a91fff1f385 100644
@@ -406,6 +406,9 @@ bool mptcp_must_wake_parent(struct subflow_context *subflow)
 	if (state != TCP_ESTABLISHED)
 		return true;
 
+	if (WARN_ON_ONCE(!tcp_sk(ssk)->is_mptcp))
+		return true;
+
 	sock_owned_by_me((const struct sock *)ssk);
 
 	for (;;) {
@@ -458,6 +461,8 @@ bool mptcp_must_wake_parent(struct subflow_context *subflow)
 		} else {
 			ack_seq = get_mapped_dsn(subflow);
 
+			WARN_ON_ONCE(!subflow->map_valid);
+
 			if (before64(ack_seq, old_ack)) {
 				/* Mapping covers data already received,
 				 * discard data in the current mapping
@@ -1170,6 +1175,9 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,
 		return ret;
 	}
 
+	sock_owned_by_me((const struct sock *)msk);
+	sock_poll_wait(file, sock, wait);
+
 	mptcp_for_each_subflow(msk, subflow) {
 		struct socket *tcp_sock;
 
Signature · 1 line
2.21.0
author
#5 lore raw

RFC PATCH 5/6 net: tcp: add __tcp_poll helper

In reply to Florian Westphal

MPTCP will need to be able to call tcp_poll to check if the tcp socket is available for reading/writing, but without adding the task to the waitlist.

This splits tcp_poll into two functions so __tcp_poll can be called by mptcp in followup patch.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 include/net/tcp.h |  1 +
 net/ipv4/tcp.c    | 12 ++++++++----
 2 files changed, 9 insertions(+), 4 deletions(-)
include/net/tcp.h +1 -0
index 3994e6b0db27..53720bb66a93 100644
@@ -396,6 +396,7 @@ void tcp_init_sock(struct sock *sk);
 void tcp_init_transfer(struct sock *sk, int bpf_op);
 __poll_t tcp_poll(struct file *file, struct socket *sock,
 		      struct poll_table_struct *wait);
+__poll_t __tcp_poll(struct sock *sk);
 int tcp_getsockopt(struct sock *sk, int level, int optname,
 		   char __user *optval, int __user *optlen);
 int tcp_setsockopt(struct sock *sk, int level, int optname,
net/ipv4/tcp.c +8 -4
index 1cd8ecf8241e..6ab9d51378ff 100644
@@ -490,15 +490,12 @@ static inline bool tcp_stream_is_readable(const struct tcp_sock *tp,
  *	take care of normal races (between the test and the event) and we don't
  *	go look at any of the socket buffers directly.
  */
-__poll_t tcp_poll(struct file *file, struct socket *sock, poll_table *wait)
+__poll_t __tcp_poll(struct sock *sk)
 {
 	__poll_t mask;
-	struct sock *sk = sock->sk;
 	const struct tcp_sock *tp = tcp_sk(sk);
 	int state;
 
-	sock_poll_wait(file, sock, wait);
-
 	state = inet_sk_state_load(sk);
 	if (state == TCP_LISTEN)
 		return inet_csk_listen_poll(sk);
@@ -590,6 +587,13 @@ __poll_t tcp_poll(struct file *file, struct socket *sock, poll_table *wait)
 
 	return mask;
 }
+
+__poll_t tcp_poll(struct file *file, struct socket *sock, poll_table *wait)
+{
+	sock_poll_wait(file, sock, wait);
+
+	return __tcp_poll(sock->sk);
+}
 EXPORT_SYMBOL(tcp_poll);
 
 int tcp_ioctl(struct sock *sk, int cmd, unsigned long arg)
Signature · 1 line
2.21.0
author
#6 lore raw

RFC PATCH 6/6 mptcp: use __tcp_poll instead of tcp_poll

In reply to Florian Westphal

tcp_poll places current task on a wait queue. This can cause us to block forever -- some subflows might be backup/failover and never receive any data.

Use __tcp_poll instead to avoid blocking current task again after parent mptcp socket has been woken up from the subflow layer.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
net/mptcp/protocol.c +1 -1
index 2a91fff1f385..533fe2c50d3a 100644
@@ -1182,7 +1182,7 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,
 		struct socket *tcp_sock;
 
 		tcp_sock = mptcp_subflow_tcp_socket(subflow);
-		ret |= tcp_poll(file, tcp_sock, wait);
+		ret |= __tcp_poll(tcp_sock->sk);
 	}
 	release_sock(sk);
 
Signature · 1 line
2.21.0
#7 lore raw

Re: [MPTCP] [PATCH RFC 6/6] mptcp: use __tcp_poll instead of tcp_poll

Quoted 17 lines · Florian Westphal

tcp_poll places current task on a wait queue. This can cause us to block forever -- some subflows might be backup/failover and never receive any data.

Use __tcp_poll instead to avoid blocking current task again after parent mptcp socket has been woken up from the subflow layer.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/protocol.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
net/mptcp/protocol.c +0 -0
index 2a91fff1f385..533fe2c50d3a 100644

net/mptcp/protocol.c static __poll_t mptcp_poll(struct file *file, struct socket *sock, · line 1187

Quoted 7 lines
net/mptcp/protocol.c +1 -1
@@ -1182,7 +1182,7 @@ static __poll_t mptcp_poll(struct file *file, struct socket *sock,
 		struct socket *tcp_sock;
 
 		tcp_sock = mptcp_subflow_tcp_socket(subflow);
-		ret |= tcp_poll(file, tcp_sock, wait);
+		ret |= __tcp_poll(tcp_sock->sk);
 	}
 	release_sock(sk);

This allows mptcp_poll working even without a 'struct socket'. we still relies on such struct presence for:

kernel_accept
kernel_sock_shutdown
kernel_setsockopt
kernel_getsockopt
kernel_bind
kernel_connect

I'm wondering if we should try to get rid completely of any depenency on 'struct socket'? That will fix the current issue with MP_JOIN, will reduce the memory usage and likely will improve accept time.

A quick look makes me think that the most problematic call to be addressed without using any 'struct socket' are kernel_bind() and kernel_connect(), perhaps we can use a transient 'struct socket' just for subflow_connect() ? WDYT?

Thanks,

Paolo

author
#8 lore raw

Re: [MPTCP] [PATCH RFC 6/6] mptcp: use __tcp_poll instead of tcp_poll

Quoted 8 lines · Paolo Abeni

net/mptcp/protocol.c static __poll_t mptcp_poll(struct file *file, struct socket *sock, · line 1187

Quoted 5 lines
net/mptcp/protocol.c +1 -1
 		tcp_sock = mptcp_subflow_tcp_socket(subflow);
-		ret |= tcp_poll(file, tcp_sock, wait);
+		ret |= __tcp_poll(tcp_sock->sk);
 	}
 	release_sock(sk);

This allows mptcp_poll working even without a 'struct socket'. we still relies on such struct presence for:

Thats what I though as well, but tcp_poll assumes sk->sk_socket points to the userspace-facing socket structure.

Quoted 10 lines

kernel_accept
kernel_sock_shutdown
kernel_setsockopt
kernel_getsockopt
kernel_bind
kernel_connect

I'm wondering if we should try to get rid completely of any depenency on 'struct socket'? That will fix the current issue with MP_JOIN, will reduce the memory usage and likely will improve accept time.

It will require careful audit of the stack, afaics Tcp assumes that for anythig incoming tcp sk might have been detached from usersapce already (i.e., application called close()). But for anything coming from syscalls that doesn't seem to be the case.

I started to hack up an MP_JOIN patch that uses a per-sk workqueue to defer incoming joins so that a socket can be allocated for it.

I don't like it, but it seems easier to go with that route.

#9 lore raw
In reply to Florian Westphal

Hi,

Quoted 8 lines · Florian Westphal

This patch series is in need of a bit more work, f.e. there is too much copypastry in patch 3.

However, it is functional as-is so I am sharing it to get earlier reviews.

Problem with current mptcp_poll() is that we can't block the task on each
subflow: subflow we're polling on might be idle while another one has data ready for processing.

If I read the series correctly, patch 3 is not strictly needed for the above goal: AFAICS it should help to avoid empty/blocking recvmsg() after successful poll(), right?

I think we should rework a bit recvmsg() before going on with this - see "mptcp: partial recvmsg() refactor". Perhaps even something more drastic e.g. subflow could try move skbs from ssk->sk_receive_queue to a per msk queue, ordered by mptcp seq in subflow_data_ready(). WDYT?

Cheers,

Paolo

author
#10 lore raw
Quoted 6 lines · Paolo Abeni
Quoted 5 lines · Florian Westphal

This patch series is in need of a bit more work, f.e. there is too much copypastry in patch 3.

However, it is functional as-is so I am sharing it to get earlier reviews.

Quoted 7 lines
Quoted 3 lines

Problem with current mptcp_poll() is that we can't block the task on each
subflow: subflow we're polling on might be idle while another one has data ready for processing.

If I read the series correctly, patch 3 is not strictly needed for the above goal: AFAICS it should help to avoid empty/blocking recvmsg() after successful poll(), right?

Thats right. EPOLLIN means next recv won't block, which isn't the case at this time because we do mptcp processing in recv only.

Quoted 2 lines

I think we should rework a bit recvmsg() before going on with this - see "mptcp: partial recvmsg() refactor".

Agree.

Quoted 3 lines

Perhaps even something more
drastic e.g. subflow could try move skbs from ssk->sk_receive_queue to a per msk queue, ordered by mptcp seq in subflow_data_ready(). WDYT?

Yes, that might be a good idea/needed.
We shouldn't depend on regular recv calls for mptcp processing.