Skip to contentSkip to conversation

mptcp: add reset and fastclose option support

PATCH In review Florian Westphal 3 participants 15 messages Nov 5, 2020 – Nov 12, 2020

Subsystems (3)
lore
Share & tools
lore

Florian Westphal

author
#0 lore raw

cover mptcp: add reset and fastclose option support

This series adds reset and fast close options.
Both need common infrastructure:
1. need to parse mptcp option in tcp reset packets, and making them available to the stack
2. ability to add mptcp option to tcp reset packets

First two patches move code around and add mptcp-callsite for option deciding from rst packets.

Patch 3 adds reset support (both decoding and adding them). Patch 4 adds the fastclose receive processing. Patch 5 sends a fastclose packet when userspace calls close() while there is unread data in the mptcp rx queue.

Patch 2) is not strictly need ATM, I could rework the series to remove it and only handle fastclose for the time being. Last patch could be removed as well so we only handle the rx case (there is nothing in RFC that says we must send fastclose).

Comments welcome.

author
#1 lore raw

PATCH 1/5 tcp: make two mptcp helpers available to tcp stack

needed by followup patches to add mptcp reset (and fastclose) options to tcp reset packets.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 include/net/mptcp.h  | 10 ++++++++++
 include/net/tcp.h    |  5 +++++
 net/mptcp/protocol.h | 11 -----------
 3 files changed, 15 insertions(+), 11 deletions(-)
include/net/mptcp.h +10 -0
index b6cf07143a8a..3d57607982fa 100644
@@ -78,6 +78,11 @@ static inline bool rsk_drop_req(const struct request_sock *req)
 	return tcp_rsk(req)->is_mptcp && tcp_rsk(req)->drop_req;
 }
 
+static inline struct mptcp_ext *mptcp_get_ext(struct sk_buff *skb)
+{
+	return (struct mptcp_ext *)skb_ext_find(skb, SKB_EXT_MPTCP);
+}
+
 void mptcp_space(const struct sock *ssk, int *space, int *full_space);
 bool mptcp_syn_options(struct sock *sk, const struct sk_buff *skb,
 		       unsigned int *size, struct mptcp_out_options *opts);
@@ -169,6 +174,11 @@ static inline bool rsk_drop_req(const struct request_sock *req)
 	return false;
 }
 
+static inline struct mptcp_ext *mptcp_get_ext(struct sk_buff *skb)
+{
+	return NULL;
+}
+
 static inline void mptcp_parse_option(const struct sk_buff *skb,
 				      const unsigned char *ptr, int opsize,
 				      struct tcp_options_received *opt_rx)
include/net/tcp.h +5 -0
index f3d42cb626fc..8115164e0df6 100644
@@ -2377,4 +2377,9 @@ static inline u64 tcp_transmit_time(const struct sock *sk)
 	return 0;
 }
 
+static inline __be32 mptcp_option(u8 subopt, u8 len, u8 nib, u8 field)
+{
+	return htonl((TCPOPT_MPTCP << 24) | (len << 16) | (subopt << 12) |
+		     ((nib & 0xF) << 8) | field);
+}
 #endif	/* _TCP_H */
net/mptcp/protocol.h +0 -11
index d29c6a4749eb..66bd4d096753 100644
@@ -139,12 +139,6 @@ struct mptcp_options_received {
 	u16	port;
 };
 
-static inline __be32 mptcp_option(u8 subopt, u8 len, u8 nib, u8 field)
-{
-	return htonl((TCPOPT_MPTCP << 24) | (len << 16) | (subopt << 12) |
-		     ((nib & 0xF) << 8) | field);
-}
-
 struct mptcp_addr_info {
 	sa_family_t		family;
 	__be16			port;
@@ -568,11 +562,6 @@ void mptcp_pm_nl_rm_addr_received(struct mptcp_sock *msk);
 void mptcp_pm_nl_rm_subflow_received(struct mptcp_sock *msk, u8 rm_id);
 int mptcp_pm_nl_get_local_id(struct mptcp_sock *msk, struct sock_common *skc);
 
-static inline struct mptcp_ext *mptcp_get_ext(struct sk_buff *skb)
-{
-	return (struct mptcp_ext *)skb_ext_find(skb, SKB_EXT_MPTCP);
-}
-
 void mptcp_diag_subflow_init(struct tcp_ulp_ops *ops);
 
 static inline bool __mptcp_check_fallback(const struct mptcp_sock *msk)
Signature · 1 line
2.26.2
author
#4 lore raw

PATCH 2/5 tcp: parse mptcp options contained in reset packets

In reply to Florian Westphal

In MPTCP, TCP-level resets only affect the subflow, not the logical MPTCP connection.

MPTCP supports a reset-specific TCP option to enable senders to tell the receiver why the subflow was reset, for instance due to a protocol error, because the subflow is deemed too slow, a middlebox interfering with mptcp options was detected and so on.

This allows an MPTCP receiver to make a decision to reopen the subflow at a later time or even completely disable the path.

For this to work its needed to call the mptcp option decode function when a tcp reset is received.

For this to be useful the reason code has to be propagated to the Path Manager to use it to e.g. reconnect at a later time.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 include/net/tcp.h        |  2 +-
 net/ipv4/tcp_input.c     | 13 ++++++++-----
 net/ipv4/tcp_minisocks.c |  2 +-
 3 files changed, 10 insertions(+), 7 deletions(-)
include/net/tcp.h +1 -1
index 8115164e0df6..c0fef9e9ba20 100644
@@ -610,7 +610,7 @@ void tcp_skb_collapse_tstamp(struct sk_buff *skb,
 /* tcp_input.c */
 void tcp_rearm_rto(struct sock *sk);
 void tcp_synack_rtt_meas(struct sock *sk, struct request_sock *req);
-void tcp_reset(struct sock *sk);
+void tcp_reset(struct sock *sk, struct sk_buff *skb);
 void tcp_skb_mark_lost_uncond_verify(struct tcp_sock *tp, struct sk_buff *skb);
 void tcp_fin(struct sock *sk);
 
net/ipv4/tcp_input.c +8 -5
index fb3a7750f623..fb65d3519439 100644
@@ -4217,10 +4217,13 @@ static inline bool tcp_sequence(const struct tcp_sock *tp, u32 seq, u32 end_seq)
 }
 
 /* When we get a reset we do this. */
-void tcp_reset(struct sock *sk)
+void tcp_reset(struct sock *sk, struct sk_buff *skb)
 {
 	trace_tcp_receive_reset(sk);
 
+	if (sk_is_mptcp(sk))
+		mptcp_incoming_options(sk, skb);
+
 	/* We want the right error as BSD sees it (and indeed as we do). */
 	switch (sk->sk_state) {
 	case TCP_SYN_SENT:
@@ -5603,7 +5606,7 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
 						  &tp->last_oow_ack_time))
 				tcp_send_dupack(sk, skb);
 		} else if (tcp_reset_check(sk, skb)) {
-			tcp_reset(sk);
+			tcp_reset(sk, skb);
 		}
 		goto discard;
 	}
@@ -5639,7 +5642,7 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
 		}
 
 		if (rst_seq_match)
-			tcp_reset(sk);
+			tcp_reset(sk, skb);
 		else {
 			/* Disable TFO if RST is out-of-order
 			 * and no data has been received
@@ -6076,7 +6079,7 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 		 */
 
 		if (th->rst) {
-			tcp_reset(sk);
+			tcp_reset(sk, skb);
 			goto discard;
 		}
 
@@ -6518,7 +6521,7 @@ int tcp_rcv_state_process(struct sock *sk, struct sk_buff *skb)
 			if (TCP_SKB_CB(skb)->end_seq != TCP_SKB_CB(skb)->seq &&
 			    after(TCP_SKB_CB(skb)->end_seq - th->fin, tp->rcv_nxt)) {
 				NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONDATA);
-				tcp_reset(sk);
+				tcp_reset(sk, skb);
 				return 1;
 			}
 		}
net/ipv4/tcp_minisocks.c +1 -1
index 495dda2449fe..0055ae0a3bf8 100644
@@ -801,7 +801,7 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
 		req->rsk_ops->send_reset(sk, skb);
 	} else if (fastopen) { /* received a valid RST pkt */
 		reqsk_fastopen_remove(sk, req, true);
-		tcp_reset(sk);
+		tcp_reset(sk, skb);
 	}
 	if (!fastopen) {
 		inet_csk_reqsk_queue_drop(sk, req);
Signature · 1 line
2.26.2
author
#5 lore raw

PATCH 3/5 mptcp: add mptcp reset option support

In reply to Florian Westphal

The MPTCP reset option allows to carry a mptcp-specific error code that provides more information on the nature of a connection reset.

The Reset option data received gets stored in the mptcp skb extension structure so it can be consumed by e.g. path management.

When a subflow is closed, the desired error code that should be sent to the peer is placed in the subflow context structure.

If close happens before a suitable tcp socket has been created (for example, when HMAC fails validation), the reset code can be placed in the mptcp skb extension which then gets added to the TCP reset skb.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 include/net/mptcp.h  |  6 ++++--
 include/net/tcp.h    |  3 +++
 net/ipv4/tcp_ipv4.c  | 21 ++++++++++++++++++++-
 net/ipv6/tcp_ipv6.c  | 19 +++++++++++++++++++
 net/mptcp/options.c  | 42 +++++++++++++++++++++++++++++++++++++-----
 net/mptcp/protocol.c | 12 +++++++++---
 net/mptcp/protocol.h | 18 ++++++++++++++++++
 net/mptcp/subflow.c  | 27 ++++++++++++++++++++++++---
 8 files changed, 134 insertions(+), 14 deletions(-)
include/net/mptcp.h +4 -2
index 3d57607982fa..0aed06330a25 100644
@@ -30,8 +30,8 @@ struct mptcp_ext {
 			ack64:1,
 			mpc_map:1,
 			frozen:1,
-			__unused:1;
-	/* one byte hole */
+			reset_transient:1;
+	u8		reset_reason:4;
 };
 
 struct mptcp_out_options {
@@ -50,6 +50,8 @@ struct mptcp_out_options {
 	u8 rm_id;
 	u8 join_id;
 	u8 backup;
+	u8 reset_reason:4;
+	u8 reset_transient:1;
 	u32 nonce;
 	u64 thmac;
 	u32 token;
include/net/tcp.h +3 -0
index c0fef9e9ba20..899f87346b49 100644
@@ -193,6 +193,8 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOPT_FASTOPEN_MAGIC	0xF989
 #define TCPOPT_SMC_MAGIC	0xE2D4C3D9
 
+/* MPTCP suboptions used in TCP */
+#define MPTCPOPT_RST		8
 /*
  *     TCP option lengths
  */
@@ -216,6 +218,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOLEN_MD5SIG_ALIGNED		20
 #define TCPOLEN_MSS_ALIGNED		4
 #define TCPOLEN_EXP_SMC_BASE_ALIGNED	8
+#define TCPOLEN_MPTCP_RST		4
 
 /* Flags in tp->nonagle */
 #define TCP_NAGLE_OFF		1	/* Nagle's algo is disabled */
net/ipv4/tcp_ipv4.c +20 -1
index 7352c097ae48..c96aea5514c6 100644
@@ -660,9 +660,11 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	const struct tcphdr *th = tcp_hdr(skb);
 	struct {
 		struct tcphdr th;
+		__be32 opt[(TCPOLEN_MPTCP_RST >> 2)
 #ifdef CONFIG_TCP_MD5SIG
-		__be32 opt[(TCPOLEN_MD5SIG_ALIGNED >> 2)];
+			   + (TCPOLEN_MD5SIG_ALIGNED >> 2)
 #endif
+			];
 	} rep;
 	struct ip_reply_arg arg;
 #ifdef CONFIG_TCP_MD5SIG
@@ -770,6 +772,23 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 				     ip_hdr(skb)->daddr, &rep.th);
 	}
 #endif
+	/* Can't co-exist with TCPMD5, hence check rep.opt[0] */
+	if (sk && sk_fullsock(sk) && sk_is_mptcp(sk) && rep.opt[0] == 0) {
+		const struct mptcp_ext *ext = mptcp_get_ext(skb);
+		u8 flags = 0, reason = 0;
+
+		if (ext) {
+			flags = ext->reset_transient;
+			reason = ext->reset_reason;
+		}
+
+		rep.opt[0] = mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
+					  flags, reason);
+
+		arg.iov[0].iov_len += TCPOLEN_MPTCP_RST;
+		rep.th.doff = arg.iov[0].iov_len / 4;
+	}
+
 	arg.csum = csum_tcpudp_nofold(ip_hdr(skb)->daddr,
 				      ip_hdr(skb)->saddr, /* XXX */
 				      arg.iov[0].iov_len, IPPROTO_TCP, 0);
net/ipv6/tcp_ipv6.c +19 -0
index 8db59f4e5f13..cfe8d6b4c34c 100644
@@ -868,6 +868,7 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	struct net *net = sk ? sock_net(sk) : dev_net(skb_dst(skb)->dev);
 	struct sock *ctl_sk = net->ipv6.tcp_sk;
 	unsigned int tot_len = sizeof(struct tcphdr);
+	bool mptcp_reset = false;
 	struct dst_entry *dst;
 	__be32 *topt;
 	__u32 mark = 0;
@@ -879,6 +880,11 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 		tot_len += TCPOLEN_MD5SIG_ALIGNED;
 #endif
 
+	if (rst && sk && sk_fullsock(sk) && sk_is_mptcp(sk) && !key) {
+		tot_len += TCPOLEN_MPTCP_RST;
+		mptcp_reset = true;
+	}
+
 	buff = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + tot_len,
 			 GFP_ATOMIC);
 	if (!buff)
@@ -909,6 +915,19 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 		*topt++ = htonl(tsecr);
 	}
 
+	if (mptcp_reset) {
+		const struct mptcp_ext *ext = mptcp_get_ext(skb);
+		u8 flags = 0, reason = 0;
+
+		if (ext) {
+			flags = ext->reset_transient;
+			reason = ext->reset_reason;
+		}
+
+		*topt++ = mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
+				       flags, reason);
+	}
+
 #ifdef CONFIG_TCP_MD5SIG
 	if (key) {
 		*topt++ = htonl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) |
net/mptcp/options.c +37 -5

73 lines not on this page · raw message

net/mptcp/protocol.c +9 -3
index a6bd06c724d5..71e556540161 100644
@@ -2608,14 +2608,18 @@ bool mptcp_finish_join(struct sock *ssk)
 	pr_debug("msk=%p, subflow=%p", msk, subflow);
 
 	/* mptcp socket already closing? */
-	if (!mptcp_is_fully_established(parent))
+	if (!mptcp_is_fully_established(parent)) {
+		subflow->reset_reason = MPTCP_RST_EMPTCP;
 		return false;
+	}
 
 	if (!msk->pm.server_side)
 		return true;
 
-	if (!mptcp_pm_allow_new_subflow(msk))
+	if (!mptcp_pm_allow_new_subflow(msk)) {
+		subflow->reset_reason = MPTCP_RST_EPROHIBIT;
 		return false;
+	}
 
 	/* active connections are already on conn_list, and we can't acquire
 	 * msk lock here.
@@ -2629,8 +2633,10 @@ bool mptcp_finish_join(struct sock *ssk)
 		sock_hold(ssk);
 	}
 	spin_unlock_bh(&msk->join_list_lock);
-	if (!ret)
+	if (!ret) {
+		subflow->reset_reason = MPTCP_RST_EPROHIBIT;
 		return false;
+	}
 
 	/* attach to msk socket only after we are sure he will deal with us
 	 * at close time
net/mptcp/protocol.h +18 -0
index 66bd4d096753..8a247e50d326 100644
@@ -23,6 +23,7 @@
 #define OPTION_MPTCP_ADD_ADDR	BIT(6)
 #define OPTION_MPTCP_ADD_ADDR6	BIT(7)
 #define OPTION_MPTCP_RM_ADDR	BIT(8)
+#define OPTION_MPTCP_RST	BIT(9)
 
 /* MPTCP option subtypes */
 #define MPTCPOPT_MP_CAPABLE	0
@@ -84,6 +85,18 @@
 #define MPTCP_ADDR_IPVERSION_4	4
 #define MPTCP_ADDR_IPVERSION_6	6
 
+/* MPTCP TCPRST flags */
+#define MPTCP_RST_TRANSIENT	BIT(0)
+
+/* MPTCP Reset reason codes, rfc8684 */
+#define MPTCP_RST_EUNSPEC	0
+#define MPTCP_RST_EMPTCP	1
+#define MPTCP_RST_ERESOURCE	2
+#define MPTCP_RST_EPROHIBIT	3
+#define MPTCP_RST_EWQ2BIG	4
+#define MPTCP_RST_EBADPERF	5
+#define MPTCP_RST_EMIDDLEBOX	6
+
 /* MPTCP socket flags */
 #define MPTCP_DATA_READY	0
 #define MPTCP_NOSPACE		1
@@ -109,6 +122,7 @@ struct mptcp_options_received {
 	u16	data_len;
 	u16	mp_capable : 1,
 		mp_join : 1,
+		reset : 1,
 		dss : 1,
 		add_addr : 1,
 		rm_addr : 1,
@@ -129,6 +143,8 @@ struct mptcp_options_received {
 		__unused:2;
 	u8	addr_id;
 	u8	rm_id;
+	u8	reset_reason:4;
+	u8	reset_transient:1;
 	union {
 		struct in_addr	addr;
 #if IS_ENABLED(CONFIG_MPTCP_IPV6)
@@ -365,6 +381,8 @@ struct mptcp_subflow_context {
 	u8	hmac[MPTCPOPT_HMAC_LEN];
 	u8	local_id;
 	u8	remote_id;
+	u8	reset_transient:1;
+	u8	reset_reason:4;
 
 	struct	sock *tcp_sock;	    /* tcp sk backpointer */
 	struct	sock *conn;	    /* parent mptcp_sock */
net/mptcp/subflow.c +24 -3
index 1e9a72af67dc..e0da6712a5c3 100644
@@ -325,8 +325,10 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
 	} else if (subflow->request_join) {
 		u8 hmac[SHA256_DIGEST_SIZE];
 
-		if (!mp_opt.mp_join)
+		if (!mp_opt.mp_join) {
+			subflow->reset_reason = MPTCP_RST_EMPTCP;
 			goto do_reset;
+		}
 
 		subflow->thmac = mp_opt.thmac;
 		subflow->remote_nonce = mp_opt.nonce;
@@ -335,6 +337,7 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
 
 		if (!subflow_thmac_valid(subflow)) {
 			MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_JOINACKMAC);
+			subflow->reset_reason = MPTCP_RST_EMPTCP;
 			goto do_reset;
 		}
 
@@ -356,6 +359,7 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
 	return;
 
 do_reset:
+	subflow->reset_transient = 0;
 	mptcp_subflow_reset(sk);
 }
 
@@ -505,6 +509,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 	struct mptcp_options_received mp_opt;
 	bool fallback, fallback_is_fatal;
 	struct sock *new_msk = NULL;
+	struct mptcp_ext *mpext;
 	struct sock *child;
 
 	pr_debug("listener=%p, req=%p, conn=%p", listener, req, listener->conn);
@@ -565,8 +570,15 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 		 * to reset the context to non MPTCP status.
 		 */
 		if (!ctx || fallback) {
-			if (fallback_is_fatal)
+			if (fallback_is_fatal) {
+				mpext = skb_ext_add(skb, SKB_EXT_MPTCP);
+				if (mpext) {
+					memset(mpext, 0, sizeof(*mpext));
+					mpext->reset_reason = MPTCP_RST_EMPTCP;
+				}
+
 				goto dispose_child;
+			}
 
 			subflow_drop_ctx(child);
 			goto out;
@@ -600,8 +612,15 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 			struct mptcp_sock *owner;
 
 			owner = subflow_req->msk;
-			if (!owner)
+			if (!owner) {
+				mpext = skb_ext_add(skb, SKB_EXT_MPTCP);
+				if (mpext) {
+					memset(mpext, 0, sizeof(*mpext));
+					mpext->reset_reason = MPTCP_RST_EPROHIBIT;
+				}
+
 				goto dispose_child;
+			}
 
 			/* move the msk reference ownership to the subflow */
 			subflow_req->msk = NULL;
@@ -936,6 +955,8 @@ static bool subflow_check_data_avail(struct sock *ssk)
 	smp_wmb();
 	ssk->sk_error_report(ssk);
 	tcp_set_state(ssk, TCP_CLOSE);
+	subflow->reset_transient = 0;
+	subflow->reset_reason = MPTCP_RST_EMPTCP;
 	tcp_send_active_reset(ssk, GFP_ATOMIC);
 	subflow->data_avail = 0;
 	return false;
Signature · 1 line
2.26.2
author
#10 lore raw

PATCH 4/5 mptcp: parse and act on incoming FASTCLOSE option

In reply to Florian Westphal

Add parsing of FASTCLOSE option.
If provided key matches the local one, worker gets scheduled and closes (tcp resets) all subflows. The MPTCP socket moves to closed state.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/options.c  | 16 ++++++++++++++++
 net/mptcp/protocol.c | 32 ++++++++++++++++++++++++++++++++
 net/mptcp/protocol.h |  4 ++++
 3 files changed, 52 insertions(+)
net/mptcp/options.c +16 -0
index 785a9f4e7da8..0a940687f738 100644
@@ -281,6 +281,15 @@ static void mptcp_parse_option(const struct sk_buff *skb,
 		mp_opt->rm_id = *ptr++;
 		pr_debug("RM_ADDR: id=%d", mp_opt->rm_id);
 		break;
+	case MPTCPOPT_MP_FASTCLOSE:
+		if (opsize != TCPOLEN_MPTCP_FASTCLOSE)
+			break;
+
+		ptr += 2;
+		mp_opt->rcvr_key = get_unaligned_be64(ptr);
+		ptr += 8;
+		mp_opt->fastclose = 1;
+		break;
 	case MPTCPOPT_RST:
 		if (opsize != TCPOLEN_MPTCP_RST)
 			break;
@@ -309,6 +318,7 @@ void mptcp_get_options(const struct sk_buff *skb,
 	mp_opt->mp_join = 0;
 	mp_opt->add_addr = 0;
 	mp_opt->ahmac = 0;
+	mp_opt->fastclose = 0;
 	mp_opt->port = 0;
 	mp_opt->rm_addr = 0;
 	mp_opt->dss = 0;
@@ -954,6 +964,12 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb)
 	if (!check_fully_established(msk, sk, subflow, skb, &mp_opt))
 		return;
 
+	if (mp_opt.fastclose &&
+	    msk->local_key == mp_opt.rcvr_key) {
+		WRITE_ONCE(msk->rcv_fastclose, true);
+		mptcp_schedule_work((struct sock *)msk);
+	}
+
 	if (mp_opt.add_addr && add_addr_hmac_valid(msk, &mp_opt)) {
 		struct mptcp_addr_info addr;
 
net/mptcp/protocol.c +32 -0
index 71e556540161..7e9705943813 100644
@@ -1849,6 +1849,35 @@ static bool mptcp_check_close_timeout(const struct sock *sk)
 	return true;
 }
 
+static noinline void mptcp_check_fastclose(struct mptcp_sock *msk)
+{
+	struct mptcp_subflow_context *subflow, *tmp;
+	struct sock *sk = &msk->sk.icsk_inet.sk;
+
+	if (likely(!READ_ONCE(msk->rcv_fastclose)))
+		return;
+
+	list_for_each_entry_safe(subflow, tmp, &msk->conn_list, node) {
+		struct sock *tcp_sk = mptcp_subflow_tcp_sock(subflow);
+
+		lock_sock(tcp_sk);
+		subflow->reset_transient = 0;
+		subflow->reset_reason = MPTCP_RST_EMPTCP;
+		mptcp_subflow_reset(tcp_sk);
+		release_sock(tcp_sk);
+	}
+
+	sk = (struct sock *)msk;
+
+	inet_sk_state_store(sk, TCP_CLOSE);
+	sk->sk_shutdown = SHUTDOWN_MASK;
+	smp_mb__before_atomic(); /* SHUTDOWN must be visible first */
+	set_bit(MPTCP_DATA_READY, &msk->flags);
+
+	sk->sk_state_change(sk);
+	sk_wake_async(sk, SOCK_WAKE_WAITD, POLL_HUP);
+}
+
 static void mptcp_worker(struct work_struct *work)
 {
 	struct mptcp_sock *msk = container_of(work, struct mptcp_sock, work);
@@ -1871,6 +1900,9 @@ static void mptcp_worker(struct work_struct *work)
 		__mptcp_close_subflow(msk);
 
 	__mptcp_move_skbs(msk);
+
+	mptcp_check_fastclose(msk);
+
 	if (mptcp_send_head(sk))
 		mptcp_push_pending(sk, 0);
 
net/mptcp/protocol.h +4 -0
index 8a247e50d326..d4c99e091cb9 100644
@@ -24,6 +24,7 @@
 #define OPTION_MPTCP_ADD_ADDR6	BIT(7)
 #define OPTION_MPTCP_RM_ADDR	BIT(8)
 #define OPTION_MPTCP_RST	BIT(9)
+#define OPTION_MPTCP_FASTCLOSE	BIT(10)
 
 /* MPTCP option subtypes */
 #define MPTCPOPT_MP_CAPABLE	0
@@ -59,6 +60,7 @@
 #define TCPOLEN_MPTCP_ADD_ADDR6_BASE_PORT	22
 #define TCPOLEN_MPTCP_PORT_LEN		2
 #define TCPOLEN_MPTCP_RM_ADDR_BASE	4
+#define TCPOLEN_MPTCP_FASTCLOSE		12
 
 /* MPTCP MP_JOIN flags */
 #define MPTCPOPT_BACKUP		BIT(0)
@@ -122,6 +124,7 @@ struct mptcp_options_received {
 	u16	data_len;
 	u16	mp_capable : 1,
 		mp_join : 1,
+		fastclose : 1,
 		reset : 1,
 		dss : 1,
 		add_addr : 1,
@@ -240,6 +243,7 @@ struct mptcp_sock {
 	bool		fully_established;
 	bool		rcv_data_fin;
 	bool		snd_data_fin_enable;
+	bool		rcv_fastclose;
 	bool		use_64bit_ack; /* Set when we received a 64-bit DSN */
 	spinlock_t	join_list_lock;
 	struct work_struct work;
Signature · 1 line
2.26.2
author
#11 lore raw

PATCH 5/5 mptcp: send fastclose if userspace closes socket with unread data

In reply to Florian Westphal

Add building & sending of FASTCLOSE option.
RFC 8684 describes two methods:

A): Host sends an ACK containing the MP_FASTCLOSE
option on one subflow [..] On all the other subflows, Host A sends a regular TCP RST to close these subflows and tears them down. [..]

R): Host A sends a RST containing the MP_FASTCLOSE option on all
subflows [..]. Host A can tear down the subflows and the connection immediately.

This implements option R) only: All subflows are re-set with FASTCLOSE.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 net/mptcp/options.c  | 35 +++++++++++++++++++++++++++++++++++
 net/mptcp/protocol.c | 32 +++++++++++++++++++++++++++++++-
 net/mptcp/protocol.h |  1 +
 3 files changed, 67 insertions(+), 1 deletion(-)
net/mptcp/options.c +35 -0
index 0a940687f738..08b60d527de0 100644
@@ -681,6 +681,31 @@ static bool mptcp_established_options_rm_addr(struct sock *sk,
 	return true;
 }
 
+static bool mptcp_fastclose(const struct mptcp_sock *msk)
+{
+	return READ_ONCE(msk->snd_fastclose);
+}
+
+static bool mptcp_established_options_fastclose(struct sock *sk,
+						unsigned int *size,
+						unsigned int remaining,
+						struct mptcp_out_options *opts)
+{
+	const struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
+
+	if (likely(!mptcp_fastclose(mptcp_sk(subflow->conn))))
+		return false;
+
+	if (remaining < TCPOLEN_MPTCP_FASTCLOSE)
+		return false;
+
+	*size = TCPOLEN_MPTCP_FASTCLOSE;
+	opts->suboptions |= OPTION_MPTCP_FASTCLOSE;
+	opts->rcvr_key = subflow->remote_key;
+
+	return true;
+}
+
 static noinline void mptcp_established_options_rst(struct sock *sk, struct sk_buff *skb,
 						   unsigned int *size,
 						   unsigned int remaining,
@@ -691,6 +716,9 @@ static noinline void mptcp_established_options_rst(struct sock *sk, struct sk_bu
 	if (remaining < TCPOLEN_MPTCP_RST)
 		return;
 
+	if (mptcp_established_options_fastclose(sk, size, remaining, opts))
+		return;
+
 	*size = TCPOLEN_MPTCP_RST;
 	opts->suboptions |= OPTION_MPTCP_RST;
 	opts->reset_transient = subflow->reset_transient;
@@ -1179,6 +1207,13 @@ void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
 		ptr += 5;
 	}
 
+	if (OPTION_MPTCP_FASTCLOSE & opts->suboptions) {
+		*ptr++ = mptcp_option(MPTCPOPT_MP_FASTCLOSE,
+				      TCPOLEN_MPTCP_FASTCLOSE, 0, 0);
+		put_unaligned_be64(opts->rcvr_key, ptr);
+		ptr += 2;
+	}
+
 	if (OPTION_MPTCP_RST & opts->suboptions)
 		*ptr++ = mptcp_option(MPTCPOPT_RST,
 				      TCPOLEN_MPTCP_RST,
net/mptcp/protocol.c +31 -1
index 7e9705943813..6b6efa00cad5 100644
@@ -2133,6 +2133,29 @@ static void __mptcp_check_send_data_fin(struct sock *sk)
 	}
 }
 
+static void __mptcp_send_fastclose(struct sock *sk)
+{
+	struct mptcp_subflow_context *subflow, *tmp;
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
+	WRITE_ONCE(msk->snd_fastclose, true);
+
+	__mptcp_flush_join_list(msk);
+	__mptcp_clear_xmit(sk);
+
+	WRITE_ONCE(msk->snd_nxt, msk->write_seq);
+
+	list_for_each_entry_safe(subflow, tmp, &msk->conn_list, node) {
+		struct sock *tcp_sk = mptcp_subflow_tcp_sock(subflow);
+
+		lock_sock(tcp_sk);
+		subflow->reset_transient = 0;
+		subflow->reset_reason = MPTCP_RST_EMPTCP;
+		mptcp_subflow_reset(tcp_sk);
+		release_sock(tcp_sk);
+	}
+}
+
 static void __mptcp_wr_shutdown(struct sock *sk)
 {
 	struct mptcp_sock *msk = mptcp_sk(sk);
@@ -2185,6 +2208,7 @@ static void mptcp_close(struct sock *sk, long timeout)
 {
 	struct mptcp_subflow_context *subflow;
 	bool do_cancel_work = false;
+	bool send_fin = false;
 
 	lock_sock(sk);
 	sk->sk_shutdown = SHUTDOWN_MASK;
@@ -2197,7 +2221,13 @@ static void mptcp_close(struct sock *sk, long timeout)
 		goto cleanup;
 	}
 
-	if (mptcp_close_state(sk))
+	send_fin = mptcp_close_state(sk);
+	if (!skb_queue_empty(&sk->sk_receive_queue)) {
+		__mptcp_send_fastclose(sk);
+		send_fin = false;
+	}
+
+	if (send_fin)
 		__mptcp_wr_shutdown(sk);
 
 	sk_stream_wait_close(sk, timeout);
net/mptcp/protocol.h +1 -0
index d4c99e091cb9..93352044bff9 100644
@@ -243,6 +243,7 @@ struct mptcp_sock {
 	bool		fully_established;
 	bool		rcv_data_fin;
 	bool		snd_data_fin_enable;
+	bool		snd_fastclose;
 	bool		rcv_fastclose;
 	bool		use_64bit_ack; /* Set when we received a 64-bit DSN */
 	spinlock_t	join_list_lock;
Signature · 1 line
2.26.2
#2 lore raw

[MPTCP] Re: [PATCH MPTCP 1/5] tcp: make two mptcp helpers available to tcp stack

In reply to Florian Westphal

Hi Florian,

Thanks for your patches.

Quoted 44 lines · Florian Westphal <fw(a)strlen.de> 于2020年11月6日周五 上午1:01写道:

needed by followup patches to add mptcp reset (and fastclose) options to tcp reset packets.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 include/net/mptcp.h  | 10 ++++++++++
 include/net/tcp.h    |  5 +++++
 net/mptcp/protocol.h | 11 -----------
 3 files changed, 15 insertions(+), 11 deletions(-)
include/net/mptcp.h +10 -0
index b6cf07143a8a..3d57607982fa 100644
@@ -78,6 +78,11 @@ static inline bool rsk_drop_req(const struct request_sock *req)
        return tcp_rsk(req)->is_mptcp && tcp_rsk(req)->drop_req;
 }
 
+static inline struct mptcp_ext *mptcp_get_ext(struct sk_buff *skb)
+{
+       return (struct mptcp_ext *)skb_ext_find(skb, SKB_EXT_MPTCP);
+}
+
 void mptcp_space(const struct sock *ssk, int *space, int *full_space);
 bool mptcp_syn_options(struct sock *sk, const struct sk_buff *skb,
                       unsigned int *size, struct mptcp_out_options *opts);
@@ -169,6 +174,11 @@ static inline bool rsk_drop_req(const struct request_sock *req)
        return false;
 }
 
+static inline struct mptcp_ext *mptcp_get_ext(struct sk_buff *skb)
+{
+       return NULL;
+}
+
 static inline void mptcp_parse_option(const struct sk_buff *skb,
                                      const unsigned char *ptr, int opsize,
                                      struct tcp_options_received *opt_rx)
include/net/tcp.h +0 -0
index f3d42cb626fc..8115164e0df6 100644

include/net/tcp.h static inline u64 tcp_transmit_time(const struct sock *sk) · line 2384

Quoted 8 lines
include/net/tcp.h +5 -0
@@ -2377,4 +2377,9 @@ static inline u64 tcp_transmit_time(const struct sock *sk)
        return 0;
 }
 
+static inline __be32 mptcp_option(u8 subopt, u8 len, u8 nib, u8 field)
+{
+       return htonl((TCPOPT_MPTCP << 24) | (len << 16) | (subopt << 12) |
+                    ((nib & 0xF) << 8) | field);
+}

I think we should avoid adding code in tcp.h, should it be better to define a wrapper function of mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST, ..., ...) in net/mptcp/options.c and export it in include/net/mptcp.h? ... ...

Quoted 35 lines
include/net/tcp.h +0 -0
 #endif /* _TCP_H */
net/mptcp/protocol.h +0 -12
index d29c6a4749eb..66bd4d096753 100644
@@ -139,12 +139,6 @@ struct mptcp_options_received {
        u16     port;
 };
 
-static inline __be32 mptcp_option(u8 subopt, u8 len, u8 nib, u8 field)
-{
-       return htonl((TCPOPT_MPTCP << 24) | (len << 16) | (subopt << 12) |
-                    ((nib & 0xF) << 8) | field);
-}
-
 struct mptcp_addr_info {
        sa_family_t             family;
        __be16                  port;
@@ -568,11 +562,6 @@ void mptcp_pm_nl_rm_addr_received(struct mptcp_sock *msk);
 void mptcp_pm_nl_rm_subflow_received(struct mptcp_sock *msk, u8 rm_id);
 int mptcp_pm_nl_get_local_id(struct mptcp_sock *msk, struct sock_common *skc);
 
-static inline struct mptcp_ext *mptcp_get_ext(struct sk_buff *skb)
-{
-       return (struct mptcp_ext *)skb_ext_find(skb, SKB_EXT_MPTCP);
-}
-
 void mptcp_diag_subflow_init(struct tcp_ulp_ops *ops);
 
 static inline bool __mptcp_check_fallback(const struct mptcp_sock *msk)
--

2.26.2
_______________________________________________
mptcp mailing list -- mptcp(a)lists.01.org
To unsubscribe send an email to mptcp-leave(a)lists.01.org

#6 lore raw

[MPTCP] Re: [PATCH MPTCP 3/5] mptcp: add mptcp reset option support

In reply to Florian Westphal
Quoted 56 lines · Florian Westphal <fw(a)strlen.de> 于2020年11月6日周五 上午1:01写道:

The MPTCP reset option allows to carry a mptcp-specific error code that provides more information on the nature of a connection reset.

The Reset option data received gets stored in the mptcp skb extension structure so it can be consumed by e.g. path management.

When a subflow is closed, the desired error code that should be sent to the peer is placed in the subflow context structure.

If close happens before a suitable tcp socket has been created (for example, when HMAC fails validation), the reset code can be placed in the mptcp skb extension which then gets added to the TCP reset skb.

Signed-off-by Florian Westphal <fw(a)strlen.de>
---
 include/net/mptcp.h  |  6 ++++--
 include/net/tcp.h    |  3 +++
 net/ipv4/tcp_ipv4.c  | 21 ++++++++++++++++++++-
 net/ipv6/tcp_ipv6.c  | 19 +++++++++++++++++++
 net/mptcp/options.c  | 42 +++++++++++++++++++++++++++++++++++++-----
 net/mptcp/protocol.c | 12 +++++++++---
 net/mptcp/protocol.h | 18 ++++++++++++++++++
 net/mptcp/subflow.c  | 27 ++++++++++++++++++++++++---
 8 files changed, 134 insertions(+), 14 deletions(-)
include/net/mptcp.h +4 -2
index 3d57607982fa..0aed06330a25 100644
@@ -30,8 +30,8 @@ struct mptcp_ext {
                        ack64:1,
                        mpc_map:1,
                        frozen:1,
-                       __unused:1;
-       /* one byte hole */
+                       reset_transient:1;
+       u8              reset_reason:4;
 };
 
 struct mptcp_out_options {
@@ -50,6 +50,8 @@ struct mptcp_out_options {
        u8 rm_id;
        u8 join_id;
        u8 backup;
+       u8 reset_reason:4;
+       u8 reset_transient:1;
        u32 nonce;
        u64 thmac;
        u32 token;
include/net/tcp.h +0 -0
index c0fef9e9ba20..899f87346b49 100644

include/net/tcp.h void tcp_time_wait(struct sock *sk, int state, int timeo); · line 197

Quoted 5 lines
include/net/tcp.h +2 -0
@@ -193,6 +193,8 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOPT_FASTOPEN_MAGIC  0xF989
 #define TCPOPT_SMC_MAGIC       0xE2D4C3D9
 
+/* MPTCP suboptions used in TCP */
+#define MPTCPOPT_RST           8

And define MPTCPOPT_RST in net/mptcp/protocol.h, together with all other MPTCP suboptions ... ...

include/net/tcp.h void tcp_time_wait(struct sock *sk, int state, int timeo); · line 221

Quoted 8 lines
include/net/tcp.h +1 -0
 /*
  *     TCP option lengths
  */
@@ -216,6 +218,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOLEN_MD5SIG_ALIGNED         20
 #define TCPOLEN_MSS_ALIGNED            4
 #define TCPOLEN_EXP_SMC_BASE_ALIGNED   8
+#define TCPOLEN_MPTCP_RST              4

And define TCPOLEN_MPTCP_RST in include/net/mptcp.h ... ...

Quoted 26 lines
include/net/tcp.h +0 -0
 /* Flags in tp->nonagle */
 #define TCP_NAGLE_OFF          1       /* Nagle's algo is disabled */
net/ipv4/tcp_ipv4.c +5 -1
index 7352c097ae48..c96aea5514c6 100644
@@ -660,9 +660,11 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
        const struct tcphdr *th = tcp_hdr(skb);
        struct {
                struct tcphdr th;
+               __be32 opt[(TCPOLEN_MPTCP_RST >> 2)
 #ifdef CONFIG_TCP_MD5SIG
-               __be32 opt[(TCPOLEN_MD5SIG_ALIGNED >> 2)];
+                          + (TCPOLEN_MD5SIG_ALIGNED >> 2)
 #endif
+                       ];
        } rep;
        struct ip_reply_arg arg;
 #ifdef CONFIG_TCP_MD5SIG
@@ -770,6 +772,23 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
                                     ip_hdr(skb)->daddr, &rep.th);
        }
 #endif
+       /* Can't co-exist with TCPMD5, hence check rep.opt[0] */
+       if (sk && sk_fullsock(sk) && sk_is_mptcp(sk) && rep.opt[0] == 0) {

net/ipv4/tcp_ipv4.c static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb) · line 786

Quoted 10 lines
net/ipv4/tcp_ipv4.c +10 -0
+               const struct mptcp_ext *ext = mptcp_get_ext(skb);
+               u8 flags = 0, reason = 0;
+
+               if (ext) {
+                       flags = ext->reset_transient;
+                       reason = ext->reset_reason;
+               }
+
+               rep.opt[0] = mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
+                                         flags, reason);

And use mptcp_option's wrapper function here ... ...

Quoted 37 lines
net/ipv4/tcp_ipv4.c +5 -0
+
+               arg.iov[0].iov_len += TCPOLEN_MPTCP_RST;
+               rep.th.doff = arg.iov[0].iov_len / 4;
+       }
+
        arg.csum = csum_tcpudp_nofold(ip_hdr(skb)->daddr,
                                      ip_hdr(skb)->saddr, /* XXX */
                                      arg.iov[0].iov_len, IPPROTO_TCP, 0);
net/ipv6/tcp_ipv6.c +7 -0
index 8db59f4e5f13..cfe8d6b4c34c 100644
@@ -868,6 +868,7 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
        struct net *net = sk ? sock_net(sk) : dev_net(skb_dst(skb)->dev);
        struct sock *ctl_sk = net->ipv6.tcp_sk;
        unsigned int tot_len = sizeof(struct tcphdr);
+       bool mptcp_reset = false;
        struct dst_entry *dst;
        __be32 *topt;
        __u32 mark = 0;
@@ -879,6 +880,11 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
                tot_len += TCPOLEN_MD5SIG_ALIGNED;
 #endif
 
+       if (rst && sk && sk_fullsock(sk) && sk_is_mptcp(sk) && !key) {
+               tot_len += TCPOLEN_MPTCP_RST;
+               mptcp_reset = true;
+       }
+
        buff = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + tot_len,
                         GFP_ATOMIC);
        if (!buff)
@@ -909,6 +915,19 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
                *topt++ = htonl(tsecr);
        }
 
+       if (mptcp_reset) {

net/ipv6/tcp_ipv6.c static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32 · line 928

Quoted 10 lines
net/ipv6/tcp_ipv6.c +10 -0
+               const struct mptcp_ext *ext = mptcp_get_ext(skb);
+               u8 flags = 0, reason = 0;
+
+               if (ext) {
+                       flags = ext->reset_transient;
+                       reason = ext->reset_reason;
+               }
+
+               *topt++ = mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
+                                      flags, reason);

And use mptcp_option's wrapper function here ... ...

The whole patch is like this:

--------

include/net/mptcp.h +4 -0
index 0aed06330a25..3bc231a7dd99 100644
@@ -34,6 +34,8 @@ struct mptcp_ext {
     u8        reset_reason:4;
 };
 
+#define TCPOLEN_MPTCP_RST        4
+
 struct mptcp_out_options {
 #if IS_ENABLED(CONFIG_MPTCP)
     u16 suboptions;
@@ -155,6 +157,8 @@ void mptcp_seq_show(struct seq_file *seq);
 int mptcp_subflow_init_cookie_req(struct request_sock *req,
                   const struct sock *sk_listener,
                   struct sk_buff *skb);
+
+__be32 mptcp_option_rst(u8 flags, u8 reason);
 #else
 
 static inline void mptcp_init(void)
@@ -240,6 +244,11 @@ static inline int

mptcp_subflow_init_cookie_req(struct request_sock *req,
{
return 0; /* TCP fallback */
}
+
+__be32 mptcp_option_rst(u8 flags, u8 reason) +{
+ return 0;
+}
#endif /* CONFIG_MPTCP */

#if IS_ENABLED(CONFIG_MPTCP_IPV6)

include/net/tcp.h +0 -8
index 899f87346b49..0a3a7ecfdc4b 100644
@@ -193,8 +193,6 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOPT_FASTOPEN_MAGIC    0xF989
 #define TCPOPT_SMC_MAGIC    0xE2D4C3D9
 
-/* MPTCP suboptions used in TCP */
-#define MPTCPOPT_RST        8
 /*
  *     TCP option lengths
  */
@@ -218,7 +216,6 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOLEN_MD5SIG_ALIGNED        20
 #define TCPOLEN_MSS_ALIGNED        4
 #define TCPOLEN_EXP_SMC_BASE_ALIGNED    8
-#define TCPOLEN_MPTCP_RST        4
 
 /* Flags in tp->nonagle */
 #define TCP_NAGLE_OFF        1    /* Nagle's algo is disabled */
@@ -2380,9 +2377,4 @@ static inline u64 tcp_transmit_time(const struct sock *sk)
     return 0;
 }
 
-static inline __be32 mptcp_option(u8 subopt, u8 len, u8 nib, u8 field)
-{
-    return htonl((TCPOPT_MPTCP << 24) | (len << 16) | (subopt << 12) |
-             ((nib & 0xF) << 8) | field);
-}
 #endif    /* _TCP_H */
net/ipv4/tcp_ipv4.c +0 -0
index c96aea5514c6..c93f820709df 100644
@@ -782,8 +782,7 @@ static void tcp_v4_send_reset(const struct sock

*sk, struct sk_buff *skb)

             reason = ext->reset_reason;
         }

- rep.opt[0] = mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
- flags, reason);
+ rep.opt[0] = mptcp_option_rst(flags, reason);

         arg.iov[0].iov_len += TCPOLEN_MPTCP_RST;
         rep.th.doff = arg.iov[0].iov_len / 4;
net/ipv6/tcp_ipv6.c +0 -0
index cfe8d6b4c34c..2f903f78051f 100644
@@ -924,8 +924,7 @@ static void tcp_v6_send_response(const struct sock

*sk, struct sk_buff *skb, u32

             reason = ext->reset_reason;
         }

- *topt++ = mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
- flags, reason);

+        *topt++ = mptcp_option_rst(flags, reason);
     }

#ifdef CONFIG_TCP_MD5SIG

net/mptcp/options.c +0 -0
index 08b60d527de0..2cda5bf46372 100644
@@ -706,6 +706,12 @@ static bool

mptcp_established_options_fastclose(struct sock *sk,

     return true;

}

+__be32 mptcp_option_rst(u8 flags, u8 reason)
+{
+ return mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
+ flags, reason);
+}
+
static noinline void mptcp_established_options_rst(struct sock *sk,
struct sk_buff *skb,
unsigned int *size,
unsigned int remaining,

net/mptcp/protocol.h +7 -0
index 93352044bff9..c7f8df418ce1 100644
@@ -35,6 +35,7 @@
 #define MPTCPOPT_MP_PRIO    5
 #define MPTCPOPT_MP_FAIL    6
 #define MPTCPOPT_MP_FASTCLOSE    7
+#define MPTCPOPT_RST        8
 
 /* MPTCP suboption lengths */
 #define TCPOLEN_MPTCP_MPC_SYN        4
@@ -158,6 +159,12 @@ struct mptcp_options_received {
     u16    port;
 };
 
+static inline __be32 mptcp_option(u8 subopt, u8 len, u8 nib, u8 field)
+{
+    return htonl((TCPOPT_MPTCP << 24) | (len << 16) | (subopt << 12) |
+             ((nib & 0xF) << 8) | field);
+}
+
 struct mptcp_addr_info {
     sa_family_t        family;
     __be16            port;

--------

I didn't tested this patch, hope it works well.

-Geliang

Quoted 269 lines
net/ipv6/tcp_ipv6.c +2 -0
+       }
+
 #ifdef CONFIG_TCP_MD5SIG
        if (key) {
                *topt++ = htonl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) |
net/mptcp/options.c +37 -5
index 248e3930c0cb..785a9f4e7da8 100644
@@ -281,7 +281,17 @@ static void mptcp_parse_option(const struct sk_buff *skb,
                mp_opt->rm_id = *ptr++;
                pr_debug("RM_ADDR: id=%d", mp_opt->rm_id);
                break;
+       case MPTCPOPT_RST:
+               if (opsize != TCPOLEN_MPTCP_RST)
+                       break;
 
+               if (!(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_RST))
+                       break;
+               mp_opt->reset = 1;
+               flags = *ptr++;
+               mp_opt->reset_transient = flags & MPTCP_RST_TRANSIENT;
+               mp_opt->reset_reason = *ptr;
+               break;
        default:
                break;
        }
@@ -302,6 +312,7 @@ void mptcp_get_options(const struct sk_buff *skb,
        mp_opt->port = 0;
        mp_opt->rm_addr = 0;
        mp_opt->dss = 0;
+       mp_opt->reset = 0;
 
        length = (th->doff * 4) - sizeof(struct tcphdr);
        ptr = (const unsigned char *)(th + 1);
@@ -660,6 +671,22 @@ static bool mptcp_established_options_rm_addr(struct sock *sk,
        return true;
 }
 
+static noinline void mptcp_established_options_rst(struct sock *sk, struct sk_buff *skb,
+                                                  unsigned int *size,
+                                                  unsigned int remaining,
+                                                  struct mptcp_out_options *opts)
+{
+       const struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
+
+       if (remaining < TCPOLEN_MPTCP_RST)
+               return;
+
+       *size = TCPOLEN_MPTCP_RST;
+       opts->suboptions |= OPTION_MPTCP_RST;
+       opts->reset_transient = subflow->reset_transient;
+       opts->reset_reason = subflow->reset_reason;
+}
+
 bool mptcp_established_options(struct sock *sk, struct sk_buff *skb,
                               unsigned int *size, unsigned int remaining,
                               struct mptcp_out_options *opts)
@@ -672,11 +699,10 @@ bool mptcp_established_options(struct sock *sk, struct sk_buff *skb,
        if (unlikely(mptcp_check_fallback(sk)))
                return false;
 
-       /* prevent adding of any MPTCP related options on reset packet
-        * until we support MP_TCPRST/MP_FASTCLOSE
-        */
-       if (unlikely(skb && TCP_SKB_CB(skb)->tcp_flags & TCPHDR_RST))
-               return false;
+       if (unlikely(skb && TCP_SKB_CB(skb)->tcp_flags & TCPHDR_RST)) {
+               mptcp_established_options_rst(sk, skb, size, remaining, opts);
+               return true;
+       }
 
        if (mptcp_established_options_mp(sk, skb, &opt_size, remaining, opts))
                ret = true;
@@ -1137,6 +1163,12 @@ void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
                ptr += 5;
        }
 
+       if (OPTION_MPTCP_RST & opts->suboptions)
+               *ptr++ = mptcp_option(MPTCPOPT_RST,
+                                     TCPOLEN_MPTCP_RST,
+                                     opts->reset_transient,
+                                     opts->reset_reason);
+
        if (opts->ext_copy.use_ack || opts->ext_copy.use_map) {
                struct mptcp_ext *mpext = &opts->ext_copy;
                u8 len = TCPOLEN_MPTCP_DSS_BASE;
net/mptcp/protocol.c +9 -3
index a6bd06c724d5..71e556540161 100644
@@ -2608,14 +2608,18 @@ bool mptcp_finish_join(struct sock *ssk)
        pr_debug("msk=%p, subflow=%p", msk, subflow);
 
        /* mptcp socket already closing? */
-       if (!mptcp_is_fully_established(parent))
+       if (!mptcp_is_fully_established(parent)) {
+               subflow->reset_reason = MPTCP_RST_EMPTCP;
                return false;
+       }
 
        if (!msk->pm.server_side)
                return true;
 
-       if (!mptcp_pm_allow_new_subflow(msk))
+       if (!mptcp_pm_allow_new_subflow(msk)) {
+               subflow->reset_reason = MPTCP_RST_EPROHIBIT;
                return false;
+       }
 
        /* active connections are already on conn_list, and we can't acquire
         * msk lock here.
@@ -2629,8 +2633,10 @@ bool mptcp_finish_join(struct sock *ssk)
                sock_hold(ssk);
        }
        spin_unlock_bh(&msk->join_list_lock);
-       if (!ret)
+       if (!ret) {
+               subflow->reset_reason = MPTCP_RST_EPROHIBIT;
                return false;
+       }
 
        /* attach to msk socket only after we are sure he will deal with us
         * at close time
net/mptcp/protocol.h +18 -0
index 66bd4d096753..8a247e50d326 100644
@@ -23,6 +23,7 @@
 #define OPTION_MPTCP_ADD_ADDR  BIT(6)
 #define OPTION_MPTCP_ADD_ADDR6 BIT(7)
 #define OPTION_MPTCP_RM_ADDR   BIT(8)
+#define OPTION_MPTCP_RST       BIT(9)
 
 /* MPTCP option subtypes */
 #define MPTCPOPT_MP_CAPABLE    0
@@ -84,6 +85,18 @@
 #define MPTCP_ADDR_IPVERSION_4 4
 #define MPTCP_ADDR_IPVERSION_6 6
 
+/* MPTCP TCPRST flags */
+#define MPTCP_RST_TRANSIENT    BIT(0)
+
+/* MPTCP Reset reason codes, rfc8684 */
+#define MPTCP_RST_EUNSPEC      0
+#define MPTCP_RST_EMPTCP       1
+#define MPTCP_RST_ERESOURCE    2
+#define MPTCP_RST_EPROHIBIT    3
+#define MPTCP_RST_EWQ2BIG      4
+#define MPTCP_RST_EBADPERF     5
+#define MPTCP_RST_EMIDDLEBOX   6
+
 /* MPTCP socket flags */
 #define MPTCP_DATA_READY       0
 #define MPTCP_NOSPACE          1
@@ -109,6 +122,7 @@ struct mptcp_options_received {
        u16     data_len;
        u16     mp_capable : 1,
                mp_join : 1,
+               reset : 1,
                dss : 1,
                add_addr : 1,
                rm_addr : 1,
@@ -129,6 +143,8 @@ struct mptcp_options_received {
                __unused:2;
        u8      addr_id;
        u8      rm_id;
+       u8      reset_reason:4;
+       u8      reset_transient:1;
        union {
                struct in_addr  addr;
 #if IS_ENABLED(CONFIG_MPTCP_IPV6)
@@ -365,6 +381,8 @@ struct mptcp_subflow_context {
        u8      hmac[MPTCPOPT_HMAC_LEN];
        u8      local_id;
        u8      remote_id;
+       u8      reset_transient:1;
+       u8      reset_reason:4;
 
        struct  sock *tcp_sock;     /* tcp sk backpointer */
        struct  sock *conn;         /* parent mptcp_sock */
net/mptcp/subflow.c +24 -4
index 1e9a72af67dc..e0da6712a5c3 100644
@@ -325,8 +325,10 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
        } else if (subflow->request_join) {
                u8 hmac[SHA256_DIGEST_SIZE];
 
-               if (!mp_opt.mp_join)
+               if (!mp_opt.mp_join) {
+                       subflow->reset_reason = MPTCP_RST_EMPTCP;
                        goto do_reset;
+               }
 
                subflow->thmac = mp_opt.thmac;
                subflow->remote_nonce = mp_opt.nonce;
@@ -335,6 +337,7 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
 
                if (!subflow_thmac_valid(subflow)) {
                        MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_JOINACKMAC);
+                       subflow->reset_reason = MPTCP_RST_EMPTCP;
                        goto do_reset;
                }
 
@@ -356,6 +359,7 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
        return;
 
 do_reset:
+       subflow->reset_transient = 0;
        mptcp_subflow_reset(sk);
 }
 
@@ -505,6 +509,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
        struct mptcp_options_received mp_opt;
        bool fallback, fallback_is_fatal;
        struct sock *new_msk = NULL;
+       struct mptcp_ext *mpext;
        struct sock *child;
 
        pr_debug("listener=%p, req=%p, conn=%p", listener, req, listener->conn);
@@ -565,8 +570,15 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
                 * to reset the context to non MPTCP status.
                 */
                if (!ctx || fallback) {
-                       if (fallback_is_fatal)
+                       if (fallback_is_fatal) {
+                               mpext = skb_ext_add(skb, SKB_EXT_MPTCP);
+                               if (mpext) {
+                                       memset(mpext, 0, sizeof(*mpext));
+                                       mpext->reset_reason = MPTCP_RST_EMPTCP;
+                               }
+
                                goto dispose_child;
+                       }
 
                        subflow_drop_ctx(child);
                        goto out;
@@ -600,8 +612,15 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
                        struct mptcp_sock *owner;
 
                        owner = subflow_req->msk;
-                       if (!owner)
+                       if (!owner) {
+                               mpext = skb_ext_add(skb, SKB_EXT_MPTCP);
+                               if (mpext) {
+                                       memset(mpext, 0, sizeof(*mpext));
+                                       mpext->reset_reason = MPTCP_RST_EPROHIBIT;
+                               }
+
                                goto dispose_child;
+                       }
 
                        /* move the msk reference ownership to the subflow */
                        subflow_req->msk = NULL;
@@ -936,6 +955,8 @@ static bool subflow_check_data_avail(struct sock *ssk)
        smp_wmb();
        ssk->sk_error_report(ssk);
        tcp_set_state(ssk, TCP_CLOSE);
+       subflow->reset_transient = 0;
+       subflow->reset_reason = MPTCP_RST_EMPTCP;
        tcp_send_active_reset(ssk, GFP_ATOMIC);
        subflow->data_avail = 0;
        return false;
--

2.26.2
_______________________________________________
mptcp mailing list -- mptcp(a)lists.01.org
To unsubscribe send an email to mptcp-leave(a)lists.01.org

author
#3 lore raw

[MPTCP] Re: [PATCH MPTCP 1/5] tcp: make two mptcp helpers available to tcp stack

In reply to Geliang Tang
Quoted 9 lines · Geliang Tang

include/net/tcp.h static inline u64 tcp_transmit_time(const struct sock *sk) · line 2384

Quoted 5 lines
include/net/tcp.h +5 -0
+static inline __be32 mptcp_option(u8 subopt, u8 len, u8 nib, u8 field)
+{
+       return htonl((TCPOPT_MPTCP << 24) | (len << 16) | (subopt << 12) |
+                    ((nib & 0xF) << 8) | field);
+}

I think we should avoid adding code in tcp.h, should it be better to define a wrapper function of mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST, ..., ...) in net/mptcp/options.c and export it in include/net/mptcp.h? ... ...

Can be done but it adds a function call to create a 32 bit number.

Sending RST isn't performance critical so it could be done that way too.

#8 lore raw

[MPTCP] Re: [PATCH MPTCP 3/5] mptcp: add mptcp reset option support

In reply to Florian Westphal
Quoted 76 lines · Florian Westphal

The MPTCP reset option allows to carry a mptcp-specific error code that provides more information on the nature of a connection reset.

The Reset option data received gets stored in the mptcp skb extension structure so it can be consumed by e.g. path management.

When a subflow is closed, the desired error code that should be sent to the peer is placed in the subflow context structure.

If close happens before a suitable tcp socket has been created (for example, when HMAC fails validation), the reset code can be placed in the mptcp skb extension which then gets added to the TCP reset skb.

Signed-off-by Florian Westphal <fw(a)strlen.de>

---

include/net/mptcp.h  |  6 ++++--
include/net/tcp.h    |  3 +++
net/ipv4/tcp_ipv4.c  | 21 ++++++++++++++++++++-
net/ipv6/tcp_ipv6.c  | 19 +++++++++++++++++++
net/mptcp/options.c  | 42 +++++++++++++++++++++++++++++++++++++-----
net/mptcp/protocol.c | 12 +++++++++---
net/mptcp/protocol.h | 18 ++++++++++++++++++
net/mptcp/subflow.c  | 27 ++++++++++++++++++++++++---
8 files changed, 134 insertions(+), 14 deletions(-)
include/net/mptcp.h +4 -2
index 3d57607982fa..0aed06330a25 100644
@@ -30,8 +30,8 @@ struct mptcp_ext {
 			ack64:1,
 			mpc_map:1,
 			frozen:1,
-			__unused:1;
-	/* one byte hole */
+			reset_transient:1;
+	u8		reset_reason:4;
 };
 
 struct mptcp_out_options {
@@ -50,6 +50,8 @@ struct mptcp_out_options {
 	u8 rm_id;
 	u8 join_id;
 	u8 backup;
+	u8 reset_reason:4;
+	u8 reset_transient:1;
 	u32 nonce;
 	u64 thmac;
 	u32 token;
include/net/tcp.h +3 -0
index c0fef9e9ba20..899f87346b49 100644
@@ -193,6 +193,8 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOPT_FASTOPEN_MAGIC	0xF989
 #define TCPOPT_SMC_MAGIC	0xE2D4C3D9
 
+/* MPTCP suboptions used in TCP */
+#define MPTCPOPT_RST		8
 /*
 *     TCP option lengths
 */
@@ -216,6 +218,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOLEN_MD5SIG_ALIGNED		20
 #define TCPOLEN_MSS_ALIGNED		4
 #define TCPOLEN_EXP_SMC_BASE_ALIGNED	8
+#define TCPOLEN_MPTCP_RST		4
 
 /* Flags in tp->nonagle */
 #define TCP_NAGLE_OFF		1	/* Nagle's algo is disabled */
net/ipv4/tcp_ipv4.c +0 -0
index 7352c097ae48..c96aea5514c6 100644

net/ipv4/tcp_ipv4.c static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb) · line 663

Quoted 4 lines
net/ipv4/tcp_ipv4.c +1 -0
@@ -660,9 +660,11 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	const struct tcphdr *th = tcp_hdr(skb);
 	struct {
 		struct tcphdr th;
+		__be32 opt[(TCPOLEN_MPTCP_RST >> 2)

Works for me - doesn't seem worthwhile to ifdef out one 32-bit value on the stack if MPTCP is not enabled, and having to deal with all the preprocessor combinations of MD5SIG and MPTCP.

net/ipv4/tcp_ipv4.c static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb) · line 775

Quoted 13 lines
net/ipv4/tcp_ipv4.c +3 -1
 #ifdef CONFIG_TCP_MD5SIG
-		__be32 opt[(TCPOLEN_MD5SIG_ALIGNED >> 2)];
+			   + (TCPOLEN_MD5SIG_ALIGNED >> 2)
 #endif
+			];
 	} rep;
 	struct ip_reply_arg arg;
 #ifdef CONFIG_TCP_MD5SIG
@@ -770,6 +772,23 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 				     ip_hdr(skb)->daddr, &rep.th);
 	}
 #endif
+	/* Can't co-exist with TCPMD5, hence check rep.opt[0] */

If they can't coexist, do we need space for both? (And, yes, I know I'm asking this after saying the single 32-bit value was not a big deal above)

Quoted 94 lines
net/ipv4/tcp_ipv4.c +16 -0
+	if (sk && sk_fullsock(sk) && sk_is_mptcp(sk) && rep.opt[0] == 0) {
+		const struct mptcp_ext *ext = mptcp_get_ext(skb);
+		u8 flags = 0, reason = 0;
+
+		if (ext) {
+			flags = ext->reset_transient;
+			reason = ext->reset_reason;
+		}
+
+		rep.opt[0] = mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
+					  flags, reason);
+
+		arg.iov[0].iov_len += TCPOLEN_MPTCP_RST;
+		rep.th.doff = arg.iov[0].iov_len / 4;
+	}
+
 	arg.csum = csum_tcpudp_nofold(ip_hdr(skb)->daddr,
 				      ip_hdr(skb)->saddr, /* XXX */
 				      arg.iov[0].iov_len, IPPROTO_TCP, 0);
net/ipv6/tcp_ipv6.c +19 -0
index 8db59f4e5f13..cfe8d6b4c34c 100644
@@ -868,6 +868,7 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	struct net *net = sk ? sock_net(sk) : dev_net(skb_dst(skb)->dev);
 	struct sock *ctl_sk = net->ipv6.tcp_sk;
 	unsigned int tot_len = sizeof(struct tcphdr);
+	bool mptcp_reset = false;
 	struct dst_entry *dst;
 	__be32 *topt;
 	__u32 mark = 0;
@@ -879,6 +880,11 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 		tot_len += TCPOLEN_MD5SIG_ALIGNED;
 #endif
 
+	if (rst && sk && sk_fullsock(sk) && sk_is_mptcp(sk) && !key) {
+		tot_len += TCPOLEN_MPTCP_RST;
+		mptcp_reset = true;
+	}
+
 	buff = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + tot_len,
 			 GFP_ATOMIC);
 	if (!buff)
@@ -909,6 +915,19 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 		*topt++ = htonl(tsecr);
 	}
 
+	if (mptcp_reset) {
+		const struct mptcp_ext *ext = mptcp_get_ext(skb);
+		u8 flags = 0, reason = 0;
+
+		if (ext) {
+			flags = ext->reset_transient;
+			reason = ext->reset_reason;
+		}
+
+		*topt++ = mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
+				       flags, reason);
+	}
+
 #ifdef CONFIG_TCP_MD5SIG
 	if (key) {
 		*topt++ = htonl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) |
net/mptcp/options.c +11 -0
index 248e3930c0cb..785a9f4e7da8 100644
@@ -281,7 +281,17 @@ static void mptcp_parse_option(const struct sk_buff *skb,
 		mp_opt->rm_id = *ptr++;
 		pr_debug("RM_ADDR: id=%d", mp_opt->rm_id);
 		break;
+	case MPTCPOPT_RST:
+		if (opsize != TCPOLEN_MPTCP_RST)
+			break;
 
+		if (!(TCP_SKB_CB(skb)->tcp_flags & TCPHDR_RST))
+			break;
+		mp_opt->reset = 1;
+		flags = *ptr++;
+		mp_opt->reset_transient = flags & MPTCP_RST_TRANSIENT;
+		mp_opt->reset_reason = *ptr;
+		break;
 	default:
 		break;
 	}
@@ -302,6 +312,7 @@ void mptcp_get_options(const struct sk_buff *skb,
 	mp_opt->port = 0;
 	mp_opt->rm_addr = 0;
 	mp_opt->dss = 0;
+	mp_opt->reset = 0;
 
 	length = (th->doff * 4) - sizeof(struct tcphdr);
 	ptr = (const unsigned char *)(th + 1);

net/mptcp/options.c static bool mptcp_established_options_rm_addr(struct sock *sk, · line 674

Quoted 4 lines
net/mptcp/options.c +1 -0
@@ -660,6 +671,22 @@ static bool mptcp_established_options_rm_addr(struct sock *sk,
 	return true;
 }
 
+static noinline void mptcp_established_options_rst(struct sock *sk, struct sk_buff *skb,

I'm assuming this is 'noinline' to keep mptcp_established_options() code size slightly smaller in the cache? I'm ok with that, of course, but curious if you think (or have data) that further code size reduction on that fast path would be worthwhile.

Thanks,

Mat

Quoted 226 lines
net/mptcp/options.c +25 -5
+						   unsigned int *size,
+						   unsigned int remaining,
+						   struct mptcp_out_options *opts)
+{
+	const struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
+
+	if (remaining < TCPOLEN_MPTCP_RST)
+		return;
+
+	*size = TCPOLEN_MPTCP_RST;
+	opts->suboptions |= OPTION_MPTCP_RST;
+	opts->reset_transient = subflow->reset_transient;
+	opts->reset_reason = subflow->reset_reason;
+}
+
 bool mptcp_established_options(struct sock *sk, struct sk_buff *skb,
 			       unsigned int *size, unsigned int remaining,
 			       struct mptcp_out_options *opts)
@@ -672,11 +699,10 @@ bool mptcp_established_options(struct sock *sk, struct sk_buff *skb,
 	if (unlikely(mptcp_check_fallback(sk)))
 		return false;
 
-	/* prevent adding of any MPTCP related options on reset packet
-	 * until we support MP_TCPRST/MP_FASTCLOSE
-	 */
-	if (unlikely(skb && TCP_SKB_CB(skb)->tcp_flags & TCPHDR_RST))
-		return false;
+	if (unlikely(skb && TCP_SKB_CB(skb)->tcp_flags & TCPHDR_RST)) {
+		mptcp_established_options_rst(sk, skb, size, remaining, opts);
+		return true;
+	}
 
 	if (mptcp_established_options_mp(sk, skb, &opt_size, remaining, opts))
 		ret = true;
@@ -1137,6 +1163,12 @@ void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
 		ptr += 5;
 	}
 
+	if (OPTION_MPTCP_RST & opts->suboptions)
+		*ptr++ = mptcp_option(MPTCPOPT_RST,
+				      TCPOLEN_MPTCP_RST,
+				      opts->reset_transient,
+				      opts->reset_reason);
+
 	if (opts->ext_copy.use_ack || opts->ext_copy.use_map) {
 		struct mptcp_ext *mpext = &opts->ext_copy;
 		u8 len = TCPOLEN_MPTCP_DSS_BASE;
net/mptcp/protocol.c +9 -3
index a6bd06c724d5..71e556540161 100644
@@ -2608,14 +2608,18 @@ bool mptcp_finish_join(struct sock *ssk)
 	pr_debug("msk=%p, subflow=%p", msk, subflow);
 
 	/* mptcp socket already closing? */
-	if (!mptcp_is_fully_established(parent))
+	if (!mptcp_is_fully_established(parent)) {
+		subflow->reset_reason = MPTCP_RST_EMPTCP;
 		return false;
+	}
 
 	if (!msk->pm.server_side)
 		return true;
 
-	if (!mptcp_pm_allow_new_subflow(msk))
+	if (!mptcp_pm_allow_new_subflow(msk)) {
+		subflow->reset_reason = MPTCP_RST_EPROHIBIT;
 		return false;
+	}
 
 	/* active connections are already on conn_list, and we can't acquire
 	 * msk lock here.
@@ -2629,8 +2633,10 @@ bool mptcp_finish_join(struct sock *ssk)
 		sock_hold(ssk);
 	}
 	spin_unlock_bh(&msk->join_list_lock);
-	if (!ret)
+	if (!ret) {
+		subflow->reset_reason = MPTCP_RST_EPROHIBIT;
 		return false;
+	}
 
 	/* attach to msk socket only after we are sure he will deal with us
 	 * at close time
net/mptcp/protocol.h +18 -0
index 66bd4d096753..8a247e50d326 100644
@@ -23,6 +23,7 @@
 #define OPTION_MPTCP_ADD_ADDR	BIT(6)
 #define OPTION_MPTCP_ADD_ADDR6	BIT(7)
 #define OPTION_MPTCP_RM_ADDR	BIT(8)
+#define OPTION_MPTCP_RST	BIT(9)
 
 /* MPTCP option subtypes */
 #define MPTCPOPT_MP_CAPABLE	0
@@ -84,6 +85,18 @@
 #define MPTCP_ADDR_IPVERSION_4	4
 #define MPTCP_ADDR_IPVERSION_6	6
 
+/* MPTCP TCPRST flags */
+#define MPTCP_RST_TRANSIENT	BIT(0)
+
+/* MPTCP Reset reason codes, rfc8684 */
+#define MPTCP_RST_EUNSPEC	0
+#define MPTCP_RST_EMPTCP	1
+#define MPTCP_RST_ERESOURCE	2
+#define MPTCP_RST_EPROHIBIT	3
+#define MPTCP_RST_EWQ2BIG	4
+#define MPTCP_RST_EBADPERF	5
+#define MPTCP_RST_EMIDDLEBOX	6
+
 /* MPTCP socket flags */
 #define MPTCP_DATA_READY	0
 #define MPTCP_NOSPACE		1
@@ -109,6 +122,7 @@ struct mptcp_options_received {
 	u16	data_len;
 	u16	mp_capable : 1,
 		mp_join : 1,
+		reset : 1,
 		dss : 1,
 		add_addr : 1,
 		rm_addr : 1,
@@ -129,6 +143,8 @@ struct mptcp_options_received {
 		__unused:2;
 	u8	addr_id;
 	u8	rm_id;
+	u8	reset_reason:4;
+	u8	reset_transient:1;
 	union {
 		struct in_addr	addr;
 #if IS_ENABLED(CONFIG_MPTCP_IPV6)
@@ -365,6 +381,8 @@ struct mptcp_subflow_context {
 	u8	hmac[MPTCPOPT_HMAC_LEN];
 	u8	local_id;
 	u8	remote_id;
+	u8	reset_transient:1;
+	u8	reset_reason:4;
 
 	struct	sock *tcp_sock;	    /* tcp sk backpointer */
 	struct	sock *conn;	    /* parent mptcp_sock */
net/mptcp/subflow.c +4 -1
index 1e9a72af67dc..e0da6712a5c3 100644
@@ -325,8 +325,10 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
 	} else if (subflow->request_join) {
 		u8 hmac[SHA256_DIGEST_SIZE];
 
-		if (!mp_opt.mp_join)
+		if (!mp_opt.mp_join) {
+			subflow->reset_reason = MPTCP_RST_EMPTCP;
 			goto do_reset;
+		}
 
 		subflow->thmac = mp_opt.thmac;
 		subflow->remote_nonce = mp_opt.nonce;
@@ -335,6 +337,7 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
 
 		if (!subflow_thmac_valid(subflow)) {
 			MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_JOINACKMAC);
+			subflow->reset_reason = MPTCP_RST_EMPTCP;
 			goto do_reset;
 		}
 
@@ -356,6 +359,7 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
 	return;
 

do_reset:

net/mptcp/subflow.c +1 -0
+	subflow->reset_transient = 0;
	mptcp_subflow_reset(sk);
}
net/mptcp/subflow.c +19 -2
@@ -505,6 +509,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 	struct mptcp_options_received mp_opt;
 	bool fallback, fallback_is_fatal;
 	struct sock *new_msk = NULL;
+	struct mptcp_ext *mpext;
 	struct sock *child;
 
 	pr_debug("listener=%p, req=%p, conn=%p", listener, req, listener->conn);
@@ -565,8 +570,15 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 		 * to reset the context to non MPTCP status.
 		 */
 		if (!ctx || fallback) {
-			if (fallback_is_fatal)
+			if (fallback_is_fatal) {
+				mpext = skb_ext_add(skb, SKB_EXT_MPTCP);
+				if (mpext) {
+					memset(mpext, 0, sizeof(*mpext));
+					mpext->reset_reason = MPTCP_RST_EMPTCP;
+				}
+
 				goto dispose_child;
+			}
 
 			subflow_drop_ctx(child);
 			goto out;
@@ -600,8 +612,15 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 			struct mptcp_sock *owner;
 
 			owner = subflow_req->msk;
-			if (!owner)
+			if (!owner) {
+				mpext = skb_ext_add(skb, SKB_EXT_MPTCP);
+				if (mpext) {
+					memset(mpext, 0, sizeof(*mpext));
+					mpext->reset_reason = MPTCP_RST_EPROHIBIT;
+				}
+
 				goto dispose_child;
+			}
 
 			/* move the msk reference ownership to the subflow */
 			subflow_req->msk = NULL;
@@ -936,6 +955,8 @@ static bool subflow_check_data_avail(struct sock *ssk)
 	smp_wmb();
 	ssk->sk_error_report(ssk);
 	tcp_set_state(ssk, TCP_CLOSE);
+	subflow->reset_transient = 0;
+	subflow->reset_reason = MPTCP_RST_EMPTCP;
 	tcp_send_active_reset(ssk, GFP_ATOMIC);
 	subflow->data_avail = 0;
 	return false;
Signature · 1 line
2.26.2

--
Mat Martineau
Intel

#12 lore raw
In reply to Florian Westphal
Quoted 14 lines · Florian Westphal

This series adds reset and fast close options.
Both need common infrastructure:
1. need to parse mptcp option in tcp reset packets, and making them available to the stack
2. ability to add mptcp option to tcp reset packets

First two patches move code around and add mptcp-callsite for option deciding from rst packets.

Patch 3 adds reset support (both decoding and adding them). Patch 4 adds the fastclose receive processing. Patch 5 sends a fastclose packet when userspace calls close() while there is unread data in the mptcp rx queue.

Quoted 6 lines

Patch 2) is not strictly need ATM, I could rework the series to remove it and only handle fastclose for the time being. Last patch could be removed as well so we only handle the rx case (there is nothing in RFC that says we must send fastclose).

Comments welcome.

I had a couple of minor questions on patch 3, but nothing that would keep this from going in to the export branch for some more testing. Might be easier to coordinate with Paolo's refactor changes to the workqueue that way.

Do the selftests get any code coverage on this, for example in the new-subflow-not-allowed case? And thanks for looking at packetdrill tests for this, saw the conversation on IRC.

--
Mat Martineau
Intel

author
#9 lore raw

[MPTCP] Re: [PATCH MPTCP 3/5] mptcp: add mptcp reset option support

In reply to Mat Martineau
Quoted 8 lines · Mat Martineau
Quoted 8 lines

#ifdef CONFIG_TCP_MD5SIG

net/ipv4/tcp_ipv4.c +1 -1
-		__be32 opt[(TCPOLEN_MD5SIG_ALIGNED >> 2)];
+			   + (TCPOLEN_MD5SIG_ALIGNED >> 2)

#endif

net/ipv4/tcp_ipv4.c +1 -0
+			];
	} rep;
	struct ip_reply_arg arg;

#ifdef CONFIG_TCP_MD5SIG

Quoted 8 lines

net/ipv4/tcp_ipv4.c static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb) · line 775

Quoted 5 lines
net/ipv4/tcp_ipv4.c +1 -0
@@ -770,6 +772,23 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 				     ip_hdr(skb)->daddr, &rep.th);
 	}
 #endif
+	/* Can't co-exist with TCPMD5, hence check rep.opt[0] */

If they can't coexist, do we need space for both? (And, yes, I know I'm asking this after saying the single 32-bit value was not a big deal above)

Mhhh, that comment and ..

Quoted 1 line

net/ipv4/tcp_ipv4.c static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb) · line 776

Quoted 1 line
net/ipv4/tcp_ipv4.c +1 -0
+	if (sk && sk_fullsock(sk) && sk_is_mptcp(sk) && rep.opt[0] == 0) {

the rep.opt[0] test can be removed. I do not see how we can have sk_is_mptcp() evaluate to true while md5sig is enabled at same time.

Wrt. space -- right, we do not need both, so maybe add

#ifdef CONFIG_TCP_MD5SIG

# define TCP_RST_OPTLEN_WORDS (TCPOLEN_MD5SIG_ALIGNED >> 2)
#else
# define TCP_RST_OPTLEN_WORDS (TCPOLEN_MPTCP_RST >> 2)
#endif

and use that instead for opt[] size.

Quoted 4 lines

net/mptcp/options.c static bool mptcp_established_options_rm_addr(struct sock *sk, · line 674

Quoted 1 line
net/mptcp/options.c +1 -0
+static noinline void mptcp_established_options_rst(struct sock *sk, struct sk_buff *skb,

I'm assuming this is 'noinline' to keep mptcp_established_options() code size slightly smaller in the cache?

Yes, the idea was that this code path rarely deals with tcp reset packets so that there is no need for inlining.

Quoted 3 lines

I'm ok with that, of course, but curious
if you think (or have data) that further code size reduction on that fast path would be worthwhile.

No, I do not have data for that (and I would not mind removing the noinline).

As for making more size reductions, I do not think it makes sense to look into such micro optimizations at this time.

author
#13 lore raw
In reply to Mat Martineau
Quoted 7 lines · Mat Martineau

I had a couple of minor questions on patch 3, but nothing that would keep this from going in to the export branch for some more testing. Might be easier to coordinate with Paolo's refactor changes to the workqueue that way.

Do the selftests get any code coverage on this, for example in the new-subflow-not-allowed case?

How would you do that? I could add a mib counter and check that in the script. But other than that I do not see how to expose any of this to the script/mptcp_connect.

Quoted 2 lines

And thanks for looking at packetdrill tests for this, saw the conversation on IRC.

Yes, I plan to send a pull request for fastclose and reset.

As for further direction, I think next step is to add genl notifications so mptcpd can learn about subflows that are added/removed, the reset code could then be exposed to userspace in the 'subflow was reset' case.

author
#7 lore raw

[MPTCP] Re: [PATCH MPTCP 3/5] mptcp: add mptcp reset option support

In reply to Geliang Tang
Quoted 8 lines · Geliang Tang
Quoted 8 lines
include/net/tcp.h +0 -0
index c0fef9e9ba20..899f87346b49 100644
@@ -193,6 +193,8 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOPT_FASTOPEN_MAGIC  0xF989
 #define TCPOPT_SMC_MAGIC       0xE2D4C3D9
 
Quoted 5 lines

include/net/tcp.h void tcp_time_wait(struct sock *sk, int state, int timeo); · line 197

Quoted 2 lines
include/net/tcp.h +2 -0
+/* MPTCP suboptions used in TCP */
+#define MPTCPOPT_RST           8

And define MPTCPOPT_RST in net/mptcp/protocol.h, together with all other MPTCP suboptions ... ...

I wanted to avoid entanglements between mptcp and the tcp reset path.

Quoted 7 lines

include/net/tcp.h void tcp_time_wait(struct sock *sk, int state, int timeo); · line 221

Quoted 5 lines
include/net/tcp.h +1 -0
@@ -216,6 +218,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOLEN_MD5SIG_ALIGNED         20
 #define TCPOLEN_MSS_ALIGNED            4
 #define TCPOLEN_EXP_SMC_BASE_ALIGNED   8
+#define TCPOLEN_MPTCP_RST              4

And define TCPOLEN_MPTCP_RST in include/net/mptcp.h ... ...

Its needed for sizing of the on-stack option array.

Placing it in include/net/mptcp.h means i would need to include that from tcp stack, which then brings other baggage with it.

I wanted to avoid that.

Quoted 4 lines

include/net/tcp.h void tcp_time_wait(struct sock *sk, int state, int timeo); · line 223

Quoted 2 lines
include/net/tcp.h +2 -0
+               rep.opt[0] = mptcp_option(MPTCPOPT_RST, TCPOLEN_MPTCP_RST,
+                                         flags, reason);

And use mptcp_option's wrapper function here ... ...

I wanted to avoid a function for what is 'u32 opt[0] = magic | reason'.

I will defer to Matthieu/Mat; I do not have a strong preference.

#14 lore raw
In reply to Florian Westphal
Quoted 6 lines · Florian Westphal
Quoted 5 lines · Mat Martineau

I had a couple of minor questions on patch 3, but nothing that would keep this from going in to the export branch for some more testing. Might be easier to coordinate with Paolo's refactor changes to the workqueue that way.

Quoted 6 lines
Quoted 2 lines

Do the selftests get any code coverage on this, for example in the new-subflow-not-allowed case?

How would you do that? I could add a mib counter and check that in the script. But other than that I do not see how to expose any of this to the script/mptcp_connect.

I meant it in a different sense: not that there's a specific self test result that looks at a count of MPTCP_RST, but that there is some self test (such as denying an additional subflow due to a configured limit) that would cause MPTCP_RST to be sent and processed. That way the new code is at least running during self test and, if memory leak checks/lockdep/etc. were enabled we would at least know some portion of the MPTCP_RST code was being exercised and checked during the tests even if they aren't explictly part of the results.

I should just answer my own question by capturing the pcaps during the self tests, but I haven't had a chance to build & run the changes yet and was curious if you had already checked.

If there are MIBs that make sense for real-world use, that's great and would make for better tests, but I'm not pushing for that now.

Quoted 11 lines
Quoted 3 lines

And thanks for looking at packetdrill tests for this, saw the conversation on IRC.

Yes, I plan to send a pull request for fastclose and reset.

As for further direction, I think next step is to add genl notifications so mptcpd can learn about subflows that are added/removed, the reset code could then be exposed to userspace in the 'subflow was reset' case.

Good plan. Thanks.

--
Mat Martineau
Intel