Skip to contentSkip to conversation

mptcp: cope better with MP_JOIN failure

RFC PATCH In review Paolo Abeni 2 participants 6 messages May 7, 2020

Subsystems (3)
lore
Share & tools
lore

Paolo Abeni

author
#0 lore raw

cover mptcp: cope better with MP_JOIN failure

This is an attempt to fix the weird behavior we currently have on MP JOIN failures.
In the end I was unable to find a way to address the thing without touching tcp_check_req(), but this is an attempt to keep the impact for !MPTCP build null, and the MPTCP overhead as low as possible.

Paolo Abeni (2):
mptcp: add new sock flag to deal with join subflows
mptcp: cope better with MP_JOIN failure

 include/linux/tcp.h      |  1 +
 include/net/mptcp.h      | 17 ++++++++++-------
 net/ipv4/tcp_minisocks.c |  2 +-
 net/mptcp/protocol.c     |  7 -------
 net/mptcp/subflow.c      | 19 +++++++++++++------
 5 files changed, 25 insertions(+), 21 deletions(-)
Signature · 1 line
2.21.1
author
#1 lore raw

RFC PATCH 1/2 mptcp: add new sock flag to deal with join subflows

MP_JOIN subflows must not land into the accept queue. Currently tcp_check_req() calls an mptcp specific helper to detect such scenario.

Such helper leverages the subflow context to check for MP_JOIN subflows. We need to deal also with MP JOIN failures, even when the subflow context is not available due allocation failure.

A possible solution would be changing the syn_recv_sock() signature to allow returning a more descriptive action/ error code and deal with that in tcp_check_req().

Since the above need is MPTCP specific, this patch instead uses a TCP socket hole to add a MPTCP specific flag. Such flag is used by the MPTCP syn_recv_sock() to tell tcp_check_req() how to deal with the request socket.

This change is a no-op for !MPTCP build, and makes the MPTCP code simpler. It allows also the next patch to deal correctly with MP JOIN failure.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 include/linux/tcp.h      |  1 +
 include/net/mptcp.h      | 17 ++++++++++-------
 net/ipv4/tcp_minisocks.c |  2 +-
 net/mptcp/protocol.c     |  7 -------
 net/mptcp/subflow.c      |  2 ++
 5 files changed, 14 insertions(+), 15 deletions(-)
include/linux/tcp.h +1 -0
index 4f8159e90ce1..f9a368d16953 100644
@@ -384,6 +384,7 @@ struct tcp_sock {
 			   */
 #if IS_ENABLED(CONFIG_MPTCP)
 	bool	is_mptcp;
+	bool	drop_req;
 #endif
 
 #ifdef CONFIG_TCP_MD5SIG
include/net/mptcp.h +10 -7
index 3bce2019e4da..99657f84d10a 100644
@@ -68,6 +68,11 @@ static inline bool rsk_is_mptcp(const struct request_sock *req)
 	return tcp_rsk(req)->is_mptcp;
 }
 
+static inline bool sk_drop_req(const struct sock *sk)
+{
+	return tcp_sk(sk)->is_mptcp && tcp_sk(sk)->drop_req;
+}
+
 bool mptcp_syn_options(struct sock *sk, const struct sk_buff *skb,
 		       unsigned int *size, struct mptcp_out_options *opts);
 bool mptcp_synack_options(const struct request_sock *req, unsigned int *size,
@@ -120,8 +125,6 @@ static inline bool mptcp_skb_can_collapse(const struct sk_buff *to,
 				 skb_ext_find(from, SKB_EXT_MPTCP));
 }
 
-bool mptcp_sk_is_subflow(const struct sock *sk);
-
 void mptcp_seq_show(struct seq_file *seq);
 #else
 
@@ -139,6 +142,11 @@ static inline bool rsk_is_mptcp(const struct request_sock *req)
 	return false;
 }
 
+static inline bool sk_drop_req(const struct sock *sk)
+{
+	return false;
+}
+
 static inline void mptcp_parse_option(const struct sk_buff *skb,
 				      const unsigned char *ptr, int opsize,
 				      struct tcp_options_received *opt_rx)
@@ -189,11 +197,6 @@ static inline bool mptcp_skb_can_collapse(const struct sk_buff *to,
 	return true;
 }
 
-static inline bool mptcp_sk_is_subflow(const struct sock *sk)
-{
-	return false;
-}
-
 static inline void mptcp_seq_show(struct seq_file *seq) { }
 #endif /* CONFIG_MPTCP */
 
net/ipv4/tcp_minisocks.c +1 -1
index 7e40322cc5ec..ba7cfb22dc79 100644
@@ -774,7 +774,7 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
 	if (!child)
 		goto listen_overflow;
 
-	if (own_req && sk_is_mptcp(child) && mptcp_sk_is_subflow(child)) {
+	if (own_req && sk_drop_req(child)) {
 		reqsk_queue_removed(&inet_csk(sk)->icsk_accept_queue, req);
 		inet_csk_reqsk_queue_drop_and_put(sk, req);
 		return child;
net/mptcp/protocol.c +0 -7
index e1f23016ed3f..a61e60e94137 100644
@@ -1638,13 +1638,6 @@ bool mptcp_finish_join(struct sock *sk)
 	return ret;
 }
 
-bool mptcp_sk_is_subflow(const struct sock *sk)
-{
-	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
-
-	return subflow->mp_join == 1;
-}
-
 static bool mptcp_memory_free(const struct sock *sk, int wake)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
net/mptcp/subflow.c +2 -0
index 4931a29a6f08..0d735e5aeb0d 100644
@@ -500,6 +500,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 			ctx->remote_key = mp_opt.sndr_key;
 			ctx->fully_established = mp_opt.mp_capable;
 			ctx->can_ack = mp_opt.mp_capable;
+			tcp_sk(child)->drop_req = false;
 		} else if (ctx->mp_join) {
 			struct mptcp_sock *owner;
 
@@ -512,6 +513,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 				goto close_child;
 
 			SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINACKRX);
+			tcp_sk(child)->drop_req = true;
 		}
 	}
 
Signature · 1 line
2.21.1
author
#2 lore raw

RFC PATCH 2/2 mptcp: cope better with MP_JOIN failure

In reply to Paolo Abeni

Currently, on MP_JOIN failure we reset the child
socket, but leave the request socket untouched.

tcp_check_req will deal with it according to the 'tcp_abort_on_overflow' sysctl value - by default the req socket will stay alive.

The above leads to inconsistent behavior on MP JOIN failure, and bad listener overflow accounting.

This patch addresses the issue leveraging the infrastructure just introduced to ask the TCP stack to drop the req on failure.

The child socket is not freed anymore by subflow_syn_recv_sock(), instead it's moved to a dead state and will be disposed by the next sock_put done by the TCP stack, so that listener overflow accounting is not affected by MP JOIN failure.

Reported-by Christoph Paasch <cpaasch(a)apple.com>
Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/subflow.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
net/mptcp/subflow.c +11 -6
index 0d735e5aeb0d..391e50a26d76 100644
@@ -476,7 +476,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 		 */
 		if (!ctx || fallback) {
 			if (fallback_is_fatal)
-				goto close_child;
+				goto dispose_child;
 
 			if (ctx) {
 				subflow_ulp_fallback(child, ctx);
@@ -506,11 +506,11 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 
 			owner = mptcp_token_get_sock(ctx->token);
 			if (!owner)
-				goto close_child;
+				goto dispose_child;
 
 			ctx->conn = (struct sock *)owner;
 			if (!mptcp_finish_join(child))
-				goto close_child;
+				goto dispose_child;
 
 			SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINACKRX);
 			tcp_sk(child)->drop_req = true;
@@ -530,11 +530,16 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 		      !mptcp_subflow_ctx(child)->conn));
 	return child;
 
-close_child:
+dispose_child:
+	tcp_sk(child)->drop_req = true;
 	tcp_send_active_reset(child, GFP_ATOMIC);
-	inet_csk_prepare_forced_close(child);
+	sock_set_flag(child, SOCK_DEAD);
+	percpu_counter_inc(sk->sk_prot->orphan_count);
+	inet_sk(child)->inet_num = 0;
 	tcp_done(child);
-	return NULL;
+
+	/* The last reference will be released by child_process */
+	return child;
 }
 
 static struct inet_connection_sock_af_ops subflow_specific;
Signature · 1 line
2.21.1
#3 lore raw
Quoted 54 lines · Paolo Abeni

Currently, on MP_JOIN failure we reset the child
socket, but leave the request socket untouched.

tcp_check_req will deal with it according to the 'tcp_abort_on_overflow' sysctl value - by default the req socket will stay alive.

The above leads to inconsistent behavior on MP JOIN failure, and bad listener overflow accounting.

This patch addresses the issue leveraging the infrastructure just introduced to ask the TCP stack to drop the req on failure.

The child socket is not freed anymore by subflow_syn_recv_sock(), instead it's moved to a dead state and will be disposed by the next sock_put done by the TCP stack, so that listener overflow accounting is not affected by MP JOIN failure.

Reported-by Christoph Paasch <cpaasch(a)apple.com>
Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/subflow.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
net/mptcp/subflow.c +3 -3
index 0d735e5aeb0d..391e50a26d76 100644
@@ -476,7 +476,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 		 */
 		if (!ctx || fallback) {
 			if (fallback_is_fatal)
-				goto close_child;
+				goto dispose_child;
 
 			if (ctx) {
 				subflow_ulp_fallback(child, ctx);
@@ -506,11 +506,11 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 
 			owner = mptcp_token_get_sock(ctx->token);
 			if (!owner)
-				goto close_child;
+				goto dispose_child;
 
 			ctx->conn = (struct sock *)owner;
 			if (!mptcp_finish_join(child))
-				goto close_child;
+				goto dispose_child;
 
 			SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINACKRX);
 			tcp_sk(child)->drop_req = true;
@@ -530,11 +530,16 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 		      !mptcp_subflow_ctx(child)->conn));

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 538

Quoted 10 lines
net/mptcp/subflow.c +5 -2
 	return child;
 
-close_child:
+dispose_child:
+	tcp_sk(child)->drop_req = true;
 	tcp_send_active_reset(child, GFP_ATOMIC);
-	inet_csk_prepare_forced_close(child);
+	sock_set_flag(child, SOCK_DEAD);
+	percpu_counter_inc(sk->sk_prot->orphan_count);
+	inet_sk(child)->inet_num = 0;

Should you move the above 3 lines to a separate function? Because, these lines are there solely to cheat inet_csk_destroy_sock and if that latter function changes, things will break.

For example, we could have:

void inet_csk_prepare_for_destroy_sock(...)

{
	sock_set_flag()
	percpu_counter_inc
	inet_num = 0;
}

And even make inet_csk_prepare_forced_close() call this function.

All of that, just to make sure that things don't break when inet_csk_destroy_sock changes.

Christoph

Quoted 11 lines
net/mptcp/subflow.c +3 -1
 	tcp_done(child);
-	return NULL;
+
+	/* The last reference will be released by child_process */
+	return child;
 }
 
 static struct inet_connection_sock_af_ops subflow_specific;
Signature · 1 line
2.21.1
author
#4 lore raw
Quoted 80 lines · Christoph Paasch
Quoted 54 lines · Paolo Abeni

Currently, on MP_JOIN failure we reset the child
socket, but leave the request socket untouched.

tcp_check_req will deal with it according to the 'tcp_abort_on_overflow' sysctl value - by default the req socket will stay alive.

The above leads to inconsistent behavior on MP JOIN failure, and bad listener overflow accounting.

This patch addresses the issue leveraging the infrastructure just introduced to ask the TCP stack to drop the req on failure.

The child socket is not freed anymore by subflow_syn_recv_sock(), instead it's moved to a dead state and will be disposed by the next sock_put done by the TCP stack, so that listener overflow accounting is not affected by MP JOIN failure.

Reported-by Christoph Paasch <cpaasch(a)apple.com>
Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/subflow.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
net/mptcp/subflow.c +3 -3
index 0d735e5aeb0d..391e50a26d76 100644
@@ -476,7 +476,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 		 */
 		if (!ctx || fallback) {
 			if (fallback_is_fatal)
-				goto close_child;
+				goto dispose_child;
 
 			if (ctx) {
 				subflow_ulp_fallback(child, ctx);
@@ -506,11 +506,11 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 
 			owner = mptcp_token_get_sock(ctx->token);
 			if (!owner)
-				goto close_child;
+				goto dispose_child;
 
 			ctx->conn = (struct sock *)owner;
 			if (!mptcp_finish_join(child))
-				goto close_child;
+				goto dispose_child;
 
 			SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINACKRX);
 			tcp_sk(child)->drop_req = true;
@@ -530,11 +530,16 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 		      !mptcp_subflow_ctx(child)->conn));

net/mptcp/subflow.c static struct sock *subflow_syn_recv_sock(const struct sock *sk, · line 538

Quoted 10 lines
net/mptcp/subflow.c +5 -2
 	return child;
 
-close_child:
+dispose_child:
+	tcp_sk(child)->drop_req = true;
 	tcp_send_active_reset(child, GFP_ATOMIC);
-	inet_csk_prepare_forced_close(child);
+	sock_set_flag(child, SOCK_DEAD);
+	percpu_counter_inc(sk->sk_prot->orphan_count);
+	inet_sk(child)->inet_num = 0;

Should you move the above 3 lines to a separate function? Because, these lines are there solely to cheat inet_csk_destroy_sock and if that latter function changes, things will break.

For example, we could have:

void inet_csk_prepare_for_destroy_sock(...)

{
	sock_set_flag()
	percpu_counter_inc
	inet_num = 0;
}
Quoted 4 lines

And even make inet_csk_prepare_forced_close() call this function.

All of that, just to make sure that things don't break when inet_csk_destroy_sock changes.

Agreed. I'll add a new patch factoring out the inet_csk_prepare_for_destroy_sock() function in include/net/inet_connection_sock.h, so that we don't need additional EXPORT_SYMBOL.

Thanks!

/P

#5 lore raw
In reply to Paolo Abeni

Looks good overall! Just had one comment on patch 2.

Christoph

Quoted 20 lines · Paolo Abeni

This is an attempt to fix the weird behavior we currently have on MP JOIN failures.
In the end I was unable to find a way to address the thing without touching tcp_check_req(), but this is an attempt to keep the impact for !MPTCP build null, and the MPTCP overhead as low as possible.

Paolo Abeni (2):
mptcp: add new sock flag to deal with join subflows
mptcp: cope better with MP_JOIN failure

 include/linux/tcp.h      |  1 +
 include/net/mptcp.h      | 17 ++++++++++-------
 net/ipv4/tcp_minisocks.c |  2 +-
 net/mptcp/protocol.c     |  7 -------
 net/mptcp/subflow.c      | 19 +++++++++++++------
 5 files changed, 25 insertions(+), 21 deletions(-)
Signature · 1 line
2.21.1