Skip to contentSkip to conversation

pm netlink: followup and fixes

PATCH In review Paolo Abeni 2 participants 18 messages Mar 4, 2020 – Mar 5, 2020

Subsystems (3)
lore
Share & tools
lore

Paolo Abeni

author
#0 lore raw

cover pm netlink: followup and fixes

This series bring some fixes and improvement related to the PM netlink, specifically:

- renames 4th_ack subflow field to 'fully_established':
this is hopefully more accurate
- implements support to enforce the maximum number of subflows per msk
- implements MP_JOIN 3rd ack retransmission and 4th ack delivery:
this [ab]use the TCP delack timer. I personally think the idea is nice and avoids changes to the core TCP stack, but I guess it could be disputed
- more accurare 'fully established' accounting as per RFC
- drop the msk level 'fully_established' field, as we can use the first subflow status without additional locking instead. This simplify core PM event handling
- adds more MP_JOIN related mib counters
- update the PM netlink to enforce a configurable maximum number of subflows per msk
- extend the pm netlink self-tests to check the above APIs
- uses amost all the above to fix the currently disable mp join self-tests and re-enable them

Note: mp join self tests are still unstable, e.g. can fail randomly, due to an old race between mptcp_token_update_accept() and mptcp_token_get_sock(): e.g. a client may legitly try to create a subflow after that the first one is fully established but before the server has accepted the initial socket:
mptcp_token_get_sock() will fail and MP_JOIN will fail too, incrementing MPTCP_MIB_JOINNOTOKEN.
I think the above should be addressed in a separate series - otherwise this one will be too big.

 include/uapi/linux/mptcp.h                      |   23 ++--
 net/mptcp/diag.c                                |    4 
 net/mptcp/mib.c                                 |    3 
 net/mptcp/mib.h                                 |    3 
 net/mptcp/options.c                             |  135 ++++++++++++++++++------
 net/mptcp/pm.c                                  |   35 ++++--
 net/mptcp/pm_netlink.c                          |   93 +++++++++++-----
 net/mptcp/protocol.c                            |   23 ++--
 net/mptcp/protocol.h                            |    8 +
 net/mptcp/subflow.c                             |    9 +
 tools/testing/selftests/net/mptcp/mptcp_join.sh |  100 ++++++++++++++---
 tools/testing/selftests/net/mptcp/pm_netlink.sh |   17 ++-
 tools/testing/selftests/net/mptcp/pm_nl_ctl.c   |   46 +++++---
 13 files changed, 363 insertions(+), 136 deletions(-)
author
#1 lore raw

PATCH 1/1 mptcp: rename fourth ack field

The name is misleading, it actually track the 'fully established' status.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/options.c  | 18 +++++++++---------
 net/mptcp/protocol.h |  2 +-
 net/mptcp/subflow.c  |  2 +-
 3 files changed, 11 insertions(+), 11 deletions(-)
net/mptcp/options.c +9 -9
index 45acd877bef3..a45f6f97ef45 100644
@@ -259,11 +259,11 @@ static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
 	struct mptcp_ext *mpext;
 	unsigned int data_len;
 
-	pr_debug("subflow=%p fourth_ack=%d seq=%x:%x remaining=%d", subflow,
-		 subflow->fourth_ack, subflow->snd_isn,
+	pr_debug("subflow=%p fully established=%d seq=%x:%x remaining=%d",
+		 subflow, subflow->fully_established, subflow->snd_isn,
 		 skb ? TCP_SKB_CB(skb)->seq : 0, remaining);
 
-	if (subflow->mp_capable && !subflow->fourth_ack && skb &&
+	if (subflow->mp_capable && !subflow->fully_established && skb &&
 	    subflow->snd_isn == TCP_SKB_CB(skb)->seq) {
 		/* When skb is not available, we better over-estimate the
 		 * emitted options len. A full DSS option is longer than
@@ -424,19 +424,19 @@ bool mptcp_synack_options(const struct request_sock *req, unsigned int *size,
 	return false;
 }
 
-static bool check_fourth_ack(struct mptcp_subflow_context *subflow,
-			     struct sk_buff *skb,
-			     struct mptcp_options_received *mp_opt)
+static bool check_fully_established(struct mptcp_subflow_context *subflow,
+				    struct sk_buff *skb,
+				    struct mptcp_options_received *mp_opt)
 {
 	/* here we can process OoO, in-window pkts, only in-sequence 4th ack
 	 * are relevant
 	 */
-	if (likely(subflow->fourth_ack ||
+	if (likely(subflow->fully_established ||
 		   TCP_SKB_CB(skb)->seq != subflow->ssn_offset + 1))
 		return true;
 
 	if (mp_opt->use_ack)
-		subflow->fourth_ack = 1;
+		subflow->fully_established = 1;
 
 	if (subflow->can_ack)
 		return true;
@@ -462,7 +462,7 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 	struct mptcp_ext *mpext;
 
 	mp_opt = &opt_rx->mptcp;
-	if (!check_fourth_ack(subflow, skb, mp_opt))
+	if (!check_fully_established(subflow, skb, mp_opt))
 		return;
 
 	if (!mp_opt->dss)
net/mptcp/protocol.h +1 -1
index 6c0b2c8ab674..fcba0e8f45f2 100644
@@ -119,7 +119,7 @@ struct mptcp_subflow_context {
 	u32	map_data_len;
 	u32	request_mptcp : 1,  /* send MP_CAPABLE */
 		mp_capable : 1,	    /* remote is MPTCP capable */
-		fourth_ack : 1,	    /* send initial DSS */
+		fully_established : 1,	    /* path validated */
 		conn_finished : 1,
 		map_valid : 1,
 		mpc_map : 1,
net/mptcp/subflow.c +1 -1
index 0de2a44bdaa0..b5f7a5a90ed1 100644
@@ -794,7 +794,7 @@ static void subflow_ulp_clone(const struct request_sock *req,
 	new_ctx->tcp_state_change = old_ctx->tcp_state_change;
 	new_ctx->tcp_write_space = old_ctx->tcp_write_space;
 	new_ctx->mp_capable = 1;
-	new_ctx->fourth_ack = subflow_req->remote_key_valid;
+	new_ctx->fully_established = subflow_req->remote_key_valid;
 	new_ctx->can_ack = subflow_req->remote_key_valid;
 	new_ctx->remote_key = subflow_req->remote_key;
 	new_ctx->local_key = subflow_req->local_key;
Signature · 1 line
2.21.1
author
#2 lore raw

PATCH 2/2 Squash-to: "mptcp: Add ADD_ADDR handling"

In reply to Paolo Abeni

Need a new hook to limit the number of per msk subflows

Also add new fields to properly account the total
subflows number

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/pm.c       | 9 ++++++++-
 net/mptcp/protocol.h | 5 ++++-
 2 files changed, 12 insertions(+), 2 deletions(-)
net/mptcp/pm.c +8 -1
index e869cea22f66..f17db5517544 100644
@@ -39,6 +39,12 @@ void mptcp_pm_new_connection(struct mptcp_sock *msk, int server_side)
 	WRITE_ONCE(pm->server_side, server_side);
 }
 
+bool mptcp_pm_allow_new_subflow(struct mptcp_sock *msk)
+{
+	pr_debug("msk=%p", msk);
+	return false;
+}
+
 void mptcp_pm_fully_established(struct mptcp_sock *msk)
 {
 	pr_debug("msk=%p", msk);
@@ -88,10 +94,11 @@ void mptcp_pm_data_init(struct mptcp_sock *msk)
 	msk->pm.add_addr_signaled = 0;
 	msk->pm.add_addr_accepted = 0;
 	msk->pm.local_addr_used = 0;
+	msk->pm.subflows = 0;
 	WRITE_ONCE(msk->pm.work_pending, false);
 	WRITE_ONCE(msk->pm.addr_signal, false);
-	WRITE_ONCE(msk->pm.fully_established, false);
 	WRITE_ONCE(msk->pm.accept_addr, false);
+	WRITE_ONCE(msk->pm.accept_subflow, false);
 	msk->pm.status = MPTCP_PM_IDLE;
 
 	spin_lock_init(&msk->pm.lock);
net/mptcp/protocol.h +4 -1
index 80af72fab808..b3a02735c796 100644
@@ -105,15 +105,17 @@ struct mptcp_pm_data {
 
 	bool		addr_signal;
 	bool		server_side;
-	bool		fully_established;
 	bool		work_pending;
 	bool		accept_addr;
+	bool		accept_subflow;
 	u8		add_addr_signaled;
 	u8		add_addr_accepted;
 	u8		local_addr_used;
+	u8		subflows;
 	u8		add_addr_signal_max;
 	u8		add_addr_accept_max;
 	u8		local_addr_max;
+	u8		subflows_max;
 	enum mptcp_pm_status status;
 
 	struct		work_struct work;
@@ -286,6 +288,7 @@ void mptcp_pm_init(void);
 void mptcp_pm_data_init(struct mptcp_sock *msk);
 void mptcp_pm_new_connection(struct mptcp_sock *msk, int server_side);
 void mptcp_pm_fully_established(struct mptcp_sock *msk);
+bool mptcp_pm_allow_new_subflow(struct mptcp_sock *msk);
 void mptcp_pm_connection_closed(struct mptcp_sock *msk);
 void mptcp_pm_subflow_established(struct mptcp_sock *msk,
 				  struct mptcp_subflow_context *subflow);
Signature · 1 line
2.21.1
author
#3 lore raw

PATCH 3/3 Squash-to: "mptcp: Add handling of incoming MP_JOIN requests"

In reply to Paolo Abeni

Send 4th ack on 3rd acj reception for MP_JOIN pkts and enforce no data packet before 4th ack.

Let the PM limit the number of per msk subflows.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/options.c  | 56 ++++++++++++++++++++++++++++++++------------
 net/mptcp/protocol.c |  7 +++---
 net/mptcp/protocol.h |  1 +
 3 files changed, 46 insertions(+), 18 deletions(-)
net/mptcp/options.c +41 -15
index 5e62cb9aafda..04365b8683db 100644
@@ -553,37 +553,66 @@ bool mptcp_synack_options(const struct request_sock *req, unsigned int *size,
 	return false;
 }
 
-static bool check_fully_established(struct mptcp_subflow_context *subflow,
+static bool check_fully_established(struct mptcp_sock *msk, struct sock *sk,
+				    struct mptcp_subflow_context *subflow,
 				    struct sk_buff *skb,
 				    struct mptcp_options_received *mp_opt)
 {
 	/* here we can process OoO, in-window pkts, only in-sequence 4th ack
-	 * are relevant
+	 * will make the subflow fully established
 	 */
-	if (likely(subflow->fully_established ||
-		   TCP_SKB_CB(skb)->seq != subflow->ssn_offset + 1))
-		return true;
+	if (likely(subflow->fully_established)) {
+		/* on passive sockets, check for 3rd ack retransmission
+		 * note that msk is always set by subflow_syn_recv_sock()
+		 * for mp_join subflows
+		 */
+		if (TCP_SKB_CB(skb)->seq == subflow->ssn_offset + 1 &&
+		    TCP_SKB_CB(skb)->end_seq == TCP_SKB_CB(skb)->seq &&
+		    subflow->mp_join && mp_opt->mp_join &&
+		    READ_ONCE(msk->pm.server_side))
+			tcp_send_ack(sk);
+		goto fully_established;
+	}
+
+	/* we should process OoO packets before the first subflow is fully
+	 * established, but not expected for MP_JOIN subflows
+	 */
+	if (TCP_SKB_CB(skb)->seq != subflow->ssn_offset + 1)
+		return subflow->mp_capable;
 
 	if (mp_opt->use_ack) {
+		/* subflows are fully established as soon as we get any
+		 * additional ack.
+		 */
 		subflow->fully_established = 1;
-		if (subflow->mp_join)
-			mptcp_pm_subflow_established(mptcp_sk(subflow->conn),
-						     subflow);
+		goto fully_established;
 	}
 
-	if (subflow->can_ack)
-		return true;
+	WARN_ON_ONCE(subflow->can_ack);
 
 	/* If the first established packet does not contain MP_CAPABLE + data
 	 * then fallback to TCP
 	 */
 	if (!mp_opt->mp_capable) {
 		subflow->mp_capable = 0;
-		tcp_sk(mptcp_subflow_tcp_sock(subflow))->is_mptcp = 0;
+		tcp_sk(sk)->is_mptcp = 0;
 		return false;
 	}
+
+	subflow->fully_established = 1;
 	subflow->remote_key = mp_opt->sndr_key;
 	subflow->can_ack = 1;
+
+fully_established:
+	/* msk can be null for MPC subflow on passive socket,  */
+	if (subflow->pm_notified || !msk)
+		return true;
+
+	subflow->pm_notified = 1;
+	if (subflow->mp_join)
+		mptcp_pm_subflow_established(msk, subflow);
+	else
+		mptcp_pm_fully_established(msk);
 	return true;
 }
 
@@ -596,7 +625,7 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 	struct mptcp_ext *mpext;
 
 	mp_opt = &opt_rx->mptcp;
-	if (!check_fully_established(subflow, skb, mp_opt))
+	if (!check_fully_established(msk, sk, subflow, skb, mp_opt))
 		return;
 
 	if (msk && mp_opt->add_addr) {
@@ -654,9 +683,6 @@ void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb,
 	}
 
 	mpext->data_fin = mp_opt->data_fin;
-
-	if (msk)
-		mptcp_pm_fully_established(msk);
 }
 
 void mptcp_write_options(__be32 *ptr, struct mptcp_out_options *opts)
net/mptcp/protocol.c +4 -3
index e10879368ab0..6129324d96e7 100644
@@ -1088,9 +1088,10 @@ bool mptcp_finish_join(struct sock *sk)
 		return false;
 
 	parent_sock = READ_ONCE(parent->sk_socket);
-	if (parent_sock) {
-		if (!sk->sk_socket)
-			mptcp_sock_graft(sk, parent_sock);
+	if (parent_sock && !sk->sk_socket) {
+		/* passive connection, attach to msk socket */
+		mptcp_sock_graft(sk, parent_sock);
+		return mptcp_pm_allow_new_subflow(msk);
 	}
 	return true;
 }
net/mptcp/protocol.h +1 -0
index f2698dfe41f8..4f1c3db26018 100644
@@ -198,6 +198,7 @@ struct mptcp_subflow_context {
 		mp_capable : 1,	    /* remote is MPTCP capable */
 		mp_join : 1,	    /* remote is JOINing */
 		fully_established : 1,	    /* path validated */
+		pm_notified : 1,    /* PM hook called for */
 		conn_finished : 1,
 		map_valid : 1,
 		mpc_map : 1,
Signature · 1 line
2.21.1
author
#4 lore raw

PATCH 4/4 Squash-to: "mptcp: Add handling of outgoing MP_JOIN requests"

In reply to Paolo Abeni

Schedule 3rd ack retransmission via delack timer, and clear it on fully established event

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/options.c | 61 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 52 insertions(+), 9 deletions(-)
net/mptcp/options.c +52 -9
index eb688b3f759f..6f01ae8efeee 100644
@@ -341,6 +341,38 @@ void mptcp_rcv_synsent(struct sock *sk)
 	}
 }
 
+/* MP_JOIN client subflow must wait for 4th ack before sending any data:
+ * TCP can't schedule delack timer before the subflow is fully established.
+ * MPTCP uses the delack timer to do 3rd ack retransmissions
+ */
+static void schedule_3rdack_retransmission(struct sock *sk)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+	struct tcp_sock *tp = tcp_sk(sk);
+	unsigned long timeout;
+
+	/* reschedule with a timeout above RTT, as we must look only for drop */
+	if (tp->srtt_us)
+		timeout = tp->srtt_us << 1;
+	else
+		timeout = TCP_TIMEOUT_INIT;
+
+	WARN_ON_ONCE(icsk->icsk_ack.pending & ICSK_ACK_TIMER);
+	icsk->icsk_ack.pending |= ICSK_ACK_SCHED | ICSK_ACK_TIMER;
+	icsk->icsk_ack.timeout = timeout;
+	sk_reset_timer(sk, &icsk->icsk_delack_timer, timeout);
+}
+
+static void clear_3rdack_retransmission(struct sock *sk)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+
+	sk_stop_timer(sk, &icsk->icsk_delack_timer);
+	icsk->icsk_ack.timeout = 0;
+	icsk->icsk_ack.ato = 0;
+	icsk->icsk_ack.pending &= ~(ICSK_ACK_SCHED | ICSK_ACK_TIMER);
+}
+
 static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
 					 unsigned int *size,
 					 unsigned int remaining,
@@ -350,12 +382,20 @@ static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
 	struct mptcp_ext *mpext;
 	unsigned int data_len;
 
-	pr_debug("subflow=%p fully established=%d seq=%x:%x remaining=%d",
-		 subflow, subflow->fully_established, subflow->snd_isn,
-		 skb ? TCP_SKB_CB(skb)->seq : 0, remaining);
+	/* When skb is not available, we better over-estimate the emitted
+	 * options len. A full DSS option (28 bytes) is longer than
+	 * TCPOLEN_MPTCP_MPC_ACK_DATA(22) or TCPOLEN_MPTCP_MPJ_ACK(24), so
+	 * let's the caller try to fit that.
+	 */
+	if (!skb)
+		return false;
+
+	/* MPC/MPJ needed only on 3rd ack packet */
+	if (subflow->snd_isn != TCP_SKB_CB(skb)->seq ||
+	    subflow->fully_established)
+		return false;
 
-	if (subflow->mp_capable && !subflow->fully_established && skb &&
-	    subflow->snd_isn == TCP_SKB_CB(skb)->seq) {
+	if (subflow->mp_capable) {
 		/* When skb is not available, we better over-estimate the
 		 * emitted options len. A full DSS option is longer than
 		 * TCPOLEN_MPTCP_MPC_ACK_DATA, so let's the caller try to fit
@@ -388,12 +428,13 @@ static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
 			 data_len);
 
 		return true;
-	} else if (subflow->mp_join && !subflow->fourth_ack) {
+	} else if (subflow->mp_join) {
 		opts->suboptions = OPTION_MPTCP_MPJ_ACK;
 		memcpy(opts->hmac, subflow->hmac, MPTCPOPT_HMAC_LEN);
 		*size = TCPOLEN_MPTCP_MPJ_ACK;
-		subflow->fourth_ack = 1;
 		pr_debug("subflow=%p", subflow);
+
+		schedule_3rdack_retransmission(sk);
 		return true;
 	}
 	return false;
@@ -633,10 +674,12 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *sk,
 		return true;
 
 	subflow->pm_notified = 1;
-	if (subflow->mp_join)
+	if (subflow->mp_join) {
+		clear_3rdack_retransmission(sk);
 		mptcp_pm_subflow_established(msk, subflow);
-	else
+	} else {
 		mptcp_pm_fully_established(msk);
+	}
 	return true;
 }
 
Signature · 1 line
2.21.1
author
#7 lore raw

PATCH 5/5 Squash-to: "mptcp: Implement path manager interface commands"

In reply to Paolo Abeni

Adapt the mptcp_pm_fully_established() hook to the new call site. Additionally avoid useless work if we don't need to announce any address /create any subflows

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/pm.c | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)
net/mptcp/pm.c +20 -6
index 174fde4071e0..a3fe2371d41b 100644
@@ -45,9 +45,23 @@ void mptcp_pm_new_connection(struct mptcp_sock *msk, int server_side)
 
 bool mptcp_pm_allow_new_subflow(struct mptcp_sock *msk)
 {
-	pr_debug("msk=%p", msk);
+	struct mptcp_pm_data *pm = &msk->pm;
+	int ret;
 
-	return false;
+	pr_debug("msk=%p subflows=%d max=%d allow=%d", msk, pm->subflows,
+		 pm->subflows_max, READ_ONCE(pm->accept_subflow));
+
+	/* try to avoid acquiring the lock below */
+	if (!READ_ONCE(pm->accept_subflow))
+		return false;
+
+	spin_lock_bh(&pm->lock);
+	ret = pm->subflows < pm->subflows_max;
+	if (ret && ++pm->subflows == pm->subflows_max)
+		WRITE_ONCE(pm->accept_subflow, false);
+	spin_unlock_bh(&pm->lock);
+
+	return ret;
 }
 
 static bool mptcp_pm_schedule_work(struct mptcp_sock *msk,
@@ -71,13 +85,13 @@ void mptcp_pm_fully_established(struct mptcp_sock *msk)
 	pr_debug("msk=%p", msk);
 
 	/* try to avoid acquiring the lock below */
-	if (READ_ONCE(pm->fully_established))
+	if (!READ_ONCE(pm->work_pending))
 		return;
 
 	spin_lock_bh(&pm->lock);
-	if (!READ_ONCE(pm->fully_established) &&
-	    mptcp_pm_schedule_work(msk, MPTCP_PM_ESTABLISHED))
-		WRITE_ONCE(pm->fully_established, true);
+
+	if (READ_ONCE(pm->work_pending))
+		mptcp_pm_schedule_work(msk, MPTCP_PM_ESTABLISHED);
 
 	spin_unlock_bh(&pm->lock);
 }
Signature · 1 line
2.21.1
author
#8 lore raw

PATCH 6/6 Squash-to: "mptcp: allow dumping subflow context to userspace"

In reply to Paolo Abeni

fix after field name change.

Additionally rename the exposed uAPI flags to be more consistent

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 include/uapi/linux/mptcp.h | 18 +++++++++---------
 net/mptcp/diag.c           |  4 ++--
 2 files changed, 11 insertions(+), 11 deletions(-)
include/uapi/linux/mptcp.h +9 -9
index d087048fb4a2..c564140d20f0 100644
@@ -4,15 +4,15 @@
 
 #include <linux/types.h>
 
-#define MPTCP_SUBFLOW_FLAG_MCAP_REM	BIT(0)
-#define MPTCP_SUBFLOW_FLAG_MCAP_LOC	BIT(1)
-#define MPTCP_SUBFLOW_FLAG_JOIN_REM	BIT(2)
-#define MPTCP_SUBFLOW_FLAG_JOIN_LOC	BIT(3)
-#define MPTCP_SUBFLOW_FLAG_BKUP_REM	BIT(4)
-#define MPTCP_SUBFLOW_FLAG_BKUP_LOC	BIT(5)
-#define MPTCP_SUBFLOW_FLAG_4THACK	BIT(6)
-#define MPTCP_SUBFLOW_FLAG_CONNECTED	BIT(7)
-#define MPTCP_SUBFLOW_FLAG_MAPVALID	BIT(8)
+#define MPTCP_SUBFLOW_FLAG_MCAP_REM		BIT(0)
+#define MPTCP_SUBFLOW_FLAG_MCAP_LOC		BIT(1)
+#define MPTCP_SUBFLOW_FLAG_JOIN_REM		BIT(2)
+#define MPTCP_SUBFLOW_FLAG_JOIN_LOC		BIT(3)
+#define MPTCP_SUBFLOW_FLAG_BKUP_REM		BIT(4)
+#define MPTCP_SUBFLOW_FLAG_BKUP_LOC		BIT(5)
+#define MPTCP_SUBFLOW_FLAG_FULLY_ESTABLISHED	BIT(6)
+#define MPTCP_SUBFLOW_FLAG_CONNECTED		BIT(7)
+#define MPTCP_SUBFLOW_FLAG_MAPVALID		BIT(8)
 
 enum {
 	MPTCP_SUBFLOW_ATTR_UNSPEC,
net/mptcp/diag.c +2 -2
index 4b6d9f7e418f..a536586742f2 100644
@@ -43,8 +43,8 @@ static int subflow_get_info(const struct sock *sk, struct sk_buff *skb)
 		flags |= MPTCP_SUBFLOW_FLAG_BKUP_REM;
 	if (sf->request_bkup)
 		flags |= MPTCP_SUBFLOW_FLAG_BKUP_LOC;
-	if (sf->fourth_ack)
-		flags |= MPTCP_SUBFLOW_FLAG_4THACK;
+	if (sf->fully_established)
+		flags |= MPTCP_SUBFLOW_FLAG_FULLY_ESTABLISHED;
 	if (sf->conn_finished)
 		flags |= MPTCP_SUBFLOW_FLAG_CONNECTED;
 	if (sf->map_valid)
Signature · 1 line
2.21.1
author
#9 lore raw

PATCH 7/7 Squash-to: "mptcp: add MIB counter infrastructure"

In reply to Paolo Abeni

Add more MP JOIN related counters, will help self-tests

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/mib.c | 3 +++
 net/mptcp/mib.h | 3 +++
 2 files changed, 6 insertions(+)
net/mptcp/mib.c +3 -0
index 4ab627e877d1..365bd574940c 100644
@@ -16,6 +16,9 @@ static const struct snmp_mib mptcp_snmp_list[] = {
 	SNMP_MIB_ITEM("MPTCPRetrans", MPTCP_MIB_RETRANSSEGS),
 	SNMP_MIB_ITEM("MPJoinNoTokenFound", MPTCP_MIB_JOINNOTOKEN),
 	SNMP_MIB_ITEM("MPJoinSynRx", MPTCP_MIB_JOINSYNRX),
+	SNMP_MIB_ITEM("MPJoinSynAckRx", MPTCP_MIB_JOINSYNACKRX),
+	SNMP_MIB_ITEM("MPJoinSynAckHMacFailure", MPTCP_MIB_JOINSYNACKMAC),
+	SNMP_MIB_ITEM("MPJoinAckRx", MPTCP_MIB_JOINACKRX),
 	SNMP_MIB_ITEM("MPJoinAckHMacFailure", MPTCP_MIB_JOINACKMAC),
 	SNMP_MIB_ITEM("DSSNotMatching", MPTCP_MIB_DSSNOMATCH),
 	SNMP_MIB_ITEM("InfiniteMapRx", MPTCP_MIB_INFINITEMAPRX),
net/mptcp/mib.h +3 -0
index d50b53e8209b..d7de340fc997 100644
@@ -9,6 +9,9 @@ enum linux_mptcp_mib_field {
 	MPTCP_MIB_RETRANSSEGS,		/* Segments retransmitted at the MPTCP-level */
 	MPTCP_MIB_JOINNOTOKEN,		/* Received MP_JOIN but the token was not found */
 	MPTCP_MIB_JOINSYNRX,		/* Received a SYN + MP_JOIN */
+	MPTCP_MIB_JOINSYNACKRX,		/* Received a SYN/ACK + MP_JOIN */
+	MPTCP_MIB_JOINSYNACKMAC,	/* HMAC was wrong on SYN/ACK + MP_JOIN */
+	MPTCP_MIB_JOINACKRX,		/* Received an ACK + MP_JOIN */
 	MPTCP_MIB_JOINACKMAC,		/* HMAC was wrong on ACK + MP_JOIN */
 	MPTCP_MIB_DSSNOMATCH,		/* Received a new mapping that did not match the previous one */
 	MPTCP_MIB_INFINITEMAPRX,	/* Received an infinite mapping */
Signature · 1 line
2.21.1
author
#10 lore raw

PATCH 8/8 Squash-to: "mptcp: increment MIB counters in a few places"

In reply to Paolo Abeni

incremente the newly introduced counters

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/subflow.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
net/mptcp/subflow.c +6 -1
index 639f3d6ac9dc..201a695a792b 100644
@@ -252,6 +252,7 @@ static void subflow_finish_connect(struct sock *sk, const struct sk_buff *skb)
 			goto do_reset;
 
 		subflow->conn_finished = 1;
+		MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_JOINSYNACKRX);
 	} else {
 do_reset:
 		tcp_send_active_reset(sk, GFP_ATOMIC);
@@ -374,8 +375,10 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 		opt_rx.mptcp.mp_join = 0;
 		mptcp_get_options(skb, &opt_rx);
 		if (!opt_rx.mptcp.mp_join ||
-		    !subflow_hmac_valid(req, &opt_rx))
+		    !subflow_hmac_valid(req, &opt_rx)) {
+			SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINACKMAC);
 			return NULL;
+		}
 	}
 
 create_child:
@@ -407,6 +410,8 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
 			ctx->conn = (struct sock *)owner;
 			if (!mptcp_finish_join(child))
 				goto close_child;
+
+			SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINACKRX);
 		}
 	}
 
Signature · 1 line
2.21.1
author
#11 lore raw

PATCH 9/9 Squash-to: "subflow: place further subflows on new 'join_list'"

In reply to Paolo Abeni

do not insert the new subflow on join list, if we are going to drop it

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 net/mptcp/protocol.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)
net/mptcp/protocol.c +10 -6
index 2db55fb1be6b..b3eb886b9488 100644
@@ -1519,6 +1519,7 @@ bool mptcp_finish_join(struct sock *sk)
 	struct mptcp_sock *msk = mptcp_sk(subflow->conn);
 	struct sock *parent = (void *)msk;
 	struct socket *parent_sock;
+	bool ret;
 
 	pr_debug("msk=%p, subflow=%p", msk, subflow);
 
@@ -1530,12 +1531,15 @@ bool mptcp_finish_join(struct sock *sk)
 	if (parent_sock && !sk->sk_socket) {
 		mptcp_sock_graft(sk, parent_sock);
 
-		/* active connections are already on conn_list */
-		spin_lock_bh(&msk->join_list_lock);
-		if (!WARN_ON_ONCE(!list_empty(&subflow->node)))
-			list_add_tail(&subflow->node, &msk->join_list);
-		spin_unlock_bh(&msk->join_list_lock);
-		return mptcp_pm_allow_new_subflow(msk);
+		ret = mptcp_pm_allow_new_subflow(msk);
+		if (ret) {
+			/* active connections are already on conn_list */
+			spin_lock_bh(&msk->join_list_lock);
+			if (!WARN_ON_ONCE(!list_empty(&subflow->node)))
+				list_add_tail(&subflow->node, &msk->join_list);
+			spin_unlock_bh(&msk->join_list_lock);
+		}
+		return ret;
 	}
 	return true;
 }
Signature · 1 line
2.21.1
author
#12 lore raw

PATCH 10/10 Squash-to: "mptcp: add netlink based PM"

In reply to Paolo Abeni

Implement support for max subflow limits, updating the APIs. Be sure to avoid any pending work if the subflows limit will block that.

Fix an issue in local address lookup: we must traverse also the join_list.

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 include/uapi/linux/mptcp.h |  5 +-
 net/mptcp/pm_netlink.c     | 93 +++++++++++++++++++++++++++-----------
 2 files changed, 69 insertions(+), 29 deletions(-)
include/uapi/linux/mptcp.h +3 -2
index 92414f3d1463..d061c5a6ae05 100644
@@ -45,6 +45,7 @@ enum mptcp_pm_attrs {
 
 	MPTCP_PM_ATTR_ADDR,				/* nested address */
 	MPTCP_PM_ATTR_RCV_ADD_ADDRS,			/* u32 */
+	MPTCP_PM_ATTR_SUBFLOWS,				/* u32 */
 
 	__MPTCP_PM_ATTR_MAX
 };
@@ -78,8 +79,8 @@ enum {
 	MPTCP_PM_CMD_DEL_ADDR,
 	MPTCP_PM_CMD_GET_ADDR,
 	MPTCP_PM_CMD_FLUSH_ADDRS,
-	MPTCP_PM_CMD_SET_RCV_ADD_ADDRS,
-	MPTCP_PM_CMD_GET_RCV_ADD_ADDRS,
+	MPTCP_PM_CMD_SET_LIMITS,
+	MPTCP_PM_CMD_GET_LIMITS,
 
 	__MPTCP_PM_CMD_AFTER_LAST
 };
net/mptcp/pm_netlink.c +66 -27

211 lines not on this page · raw message

Signature · 1 line
2.21.1
author
#13 lore raw

PATCH 11/11 Squash-to: "selftests: add PM netlink functional tests"

In reply to Paolo Abeni

update to new NL APIs

add checks for subflows limits

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 .../testing/selftests/net/mptcp/pm_netlink.sh | 17 +++++--
 tools/testing/selftests/net/mptcp/pm_nl_ctl.c | 46 ++++++++++++-------
 2 files changed, 41 insertions(+), 22 deletions(-)
tools/testing/selftests/net/mptcp/pm_netlink.sh +12 -5
index a6552eb8acbe..cfc743c47cb2 100755
@@ -69,7 +69,8 @@ check()
 }
 
 check "ip netns exec $ns1 ./pm_nl_ctl dump" "" "defaults addr list"
-check "ip netns exec $ns1 ./pm_nl_ctl accept" "accept 0" "defaults accept add addr"
+check "ip netns exec $ns1 ./pm_nl_ctl limits" "accept 0
+subflows 0" "defaults limits"
 
 ip netns exec $ns1 ./pm_nl_ctl add 10.0.1.1
 ip netns exec $ns1 ./pm_nl_ctl add 10.0.1.2 flags subflow dev lo
@@ -114,10 +115,16 @@ id 8 flags signal 10.0.1.8 " "id limit"
 ip netns exec $ns1 ./pm_nl_ctl flush
 check "ip netns exec $ns1 ./pm_nl_ctl dump" "" "flush addrs"
 
-ip netns exec $ns1 ./pm_nl_ctl accept 9
-check "ip netns exec $ns1 ./pm_nl_ctl accept" "accept 0" "above accept add_addr hard limit"
+ip netns exec $ns1 ./pm_nl_ctl limits 9 1
+check "ip netns exec $ns1 ./pm_nl_ctl limits" "accept 0
+subflows 0" "rcv addrs above hard limit"
 
-ip netns exec $ns1 ./pm_nl_ctl accept 8
-check "ip netns exec $ns1 ./pm_nl_ctl accept" "accept 8" "set accept add_addr"
+ip netns exec $ns1 ./pm_nl_ctl limits 1 9
+check "ip netns exec $ns1 ./pm_nl_ctl limits" "accept 0
+subflows 0" "subflows above hard limit"
+
+ip netns exec $ns1 ./pm_nl_ctl limits 8 8
+check "ip netns exec $ns1 ./pm_nl_ctl limits" "accept 8
+subflows 8" "set limits"
 
 exit $ret
tools/testing/selftests/net/mptcp/pm_nl_ctl.c +29 -17

104 lines not on this page · raw message

Signature · 1 line
2.21.1
author
#14 lore raw

PATCH 12/12 Squash-to: "selftests: add test-cases for MPTCP MP_JOIN"

In reply to Paolo Abeni

Add tests for subflows limit enforcing.

re-enable multiple subflows tests.

cleanup data gathering via appropriate nstat command line options

Signed-off-by Paolo Abeni <pabeni(a)redhat.com>
---
 .../testing/selftests/net/mptcp/mptcp_join.sh | 100 ++++++++++++++----
 1 file changed, 79 insertions(+), 21 deletions(-)
tools/testing/selftests/net/mptcp/mptcp_join.sh +79 -21
index 799914cd973a..a724f9e1fa50 100755
@@ -17,7 +17,6 @@ init()
 	capout=$(mktemp)
 
 	rndh=$(printf %x $sec)-$(mktemp -u XXXXXX)
-	nstat="/tmp/.nstat.$rndh"
 
 	ns1="ns1-$rndh"
 	ns2="ns2-$rndh"
@@ -53,7 +52,6 @@ init()
 cleanup_partial()
 {
 	rm -f "$capout"
-	rm -f "$nstat"
 
 	for netns in "$ns1" "$ns2"; do
 		ip netns del $netns
@@ -222,20 +220,50 @@ run_tests()
 chk_join_nr()
 {
 	local msg="$1"
-	local ns=$2
-	local nr=$3
+	local syn_nr=$2
+	local syn_ack_nr=$3
+	local ack_nr=$4
 	local count
+	local dump_stats
 
-	count=`NSTAT_HISTORY=$nstat ip netns exec $ns nstat |grep MPTcpExtMPJoinSynRx | awk '{print $2}'`
+	printf "%-36s %s" "$msg" "syn"
+	count=`ip netns exec $ns1 nstat -as | grep MPTcpExtMPJoinSynRx | awk '{print $2}'`
 	[ -z "$count" ] && count=0
-	printf "%-50s %s" "$msg"
-	if [ "$count" != "$nr" ]; then
-		echo "[fail] got $count JOIN[s] expected $nr"
+	if [ "$count" != "$syn_nr" ]; then
+		echo "[fail] got $count JOIN[s] syn expected $syn_nr"
 		ret=1
+		dump_stats=1
 	else
-		echo "[ ok ]"
+		echo -n "[ ok ]"
 	fi
 
+	echo -n " - synack"
+	count=`ip netns exec $ns2 nstat -as | grep MPTcpExtMPJoinSynAckRx | awk '{print $2}'`
+	[ -z "$count" ] && count=0
+	if [ "$count" != "$syn_ack_nr" ]; then
+		echo "[fail] got $count JOIN[s] synack expected $syn_ack_nr"
+		ret=1
+		dump_stats=1
+	else
+		echo -n "[ ok ]"
+	fi
+
+	echo -n " - ack"
+	count=`ip netns exec $ns1 nstat -as | grep MPTcpExtMPJoinAckRx | awk '{print $2}'`
+	[ -z "$count" ] && count=0
+	if [ "$count" != "$ack_nr" ]; then
+		echo "[fail] got $count JOIN[s] ack expected $ack_nr"
+		ret=1
+		dump_stats=1
+	else
+		echo "[ ok ]"
+	fi
+	if [ "${dump_stats}" = 1 ]; then
+		echo Server ns stats
+		ip netns exec $ns1 nstat -as | grep MPTcp
+		echo Client ns stats
+		ip netns exec $ns2 nstat -as | grep MPTcp
+	fi
 }
 
 sin=$(mktemp)
@@ -248,49 +276,79 @@ make_file "$sin" "server"
 trap cleanup EXIT
 
 run_tests $ns1 $ns2 10.0.1.1
-chk_join_nr "no JOIN" $ns1 "0"
+chk_join_nr "no JOIN" "0" "0" "0"
+
+# subflow limted by client
+reset
+ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
+run_tests $ns1 $ns2 10.0.1.1
+chk_join_nr "single subflow, limited by client" 0 0 0
+
+# subflow limted by server
+reset
+ip netns exec $ns2 ./pm_nl_ctl limits 0 1
+ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
+run_tests $ns1 $ns2 10.0.1.1
+chk_join_nr "single subflow, limited by server" 1 1 0
 
 # subflow
 reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 1
+ip netns exec $ns2 ./pm_nl_ctl limits 0 1
 ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
 run_tests $ns1 $ns2 10.0.1.1
-chk_join_nr "single subflow" $ns1 1
+chk_join_nr "single subflow" 1 1 1
 
 # multiple subflows
 reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 2
+ip netns exec $ns2 ./pm_nl_ctl limits 0 2
 ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
 ip netns exec $ns2 ./pm_nl_ctl add 10.0.2.2 flags subflow
 run_tests $ns1 $ns2 10.0.1.1
-chk_join_nr "multiple subflows" $ns1 2
+chk_join_nr "multiple subflows" 2 2 2
+
+
+# multiple subflows limited by serverf
+reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 1
+ip netns exec $ns2 ./pm_nl_ctl limits 0 2
+ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
+ip netns exec $ns2 ./pm_nl_ctl add 10.0.2.2 flags subflow
+run_tests $ns1 $ns2 10.0.1.1
+chk_join_nr "multiple subflows, limited by server" 2 2 1
 
 # add_address, unused
 reset
 ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal
 run_tests $ns1 $ns2 10.0.1.1
-chk_join_nr "unused signal address" $ns1 0
+chk_join_nr "unused signal address" 0 0 0
 
 # accept and use add_addr
 reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 1
+ip netns exec $ns2 ./pm_nl_ctl limits 1 1
 ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal
-ip netns exec $ns2 ./pm_nl_ctl accept 1
 run_tests $ns1 $ns2 10.0.1.1
-chk_join_nr "signal address" $ns1 1
+chk_join_nr "signal address" 1 1 1
 
 # accept and use add_addr with an additional subflow
 reset
 ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal
-ip netns exec $ns2 ./pm_nl_ctl accept 1
+ip netns exec $ns1 ./pm_nl_ctl limits 0 2
+ip netns exec $ns2 ./pm_nl_ctl limits 1 2
 ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
 run_tests $ns1 $ns2 10.0.1.1
-chk_join_nr "subflow and signal" $ns1 2
+chk_join_nr "subflow and signal" 2 2 2
 
 # accept and use add_addr with an additional subflows
 reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 3
 ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal
-ip netns exec $ns2 ./pm_nl_ctl accept 1
+ip netns exec $ns2 ./pm_nl_ctl limits 1 3
 ip netns exec $ns2 ./pm_nl_ctl add 10.0.2.2 flags subflow
-ip netns exec $ns1 ./pm_nl_ctl add 10.0.3.2 flags signal
+ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
 run_tests $ns1 $ns2 10.0.1.1
-chk_join_nr "multiple subflows and signal" $ns1 3
+chk_join_nr "multiple subflows and signal" 3 3 3
 
-exit $ret
\ No newline at end of file
+exit $ret
Signature · 1 line
2.21.1
author
#15 lore raw
In reply to Paolo Abeni
Quoted 6 lines · Paolo Abeni

Note: mp join self tests are still unstable, e.g. can fail randomly, due to an old race between mptcp_token_update_accept() and mptcp_token_get_sock(): e.g. a client may legitly try to create a subflow after that the first one is fully established but before the server has accepted the initial socket:
mptcp_token_get_sock() will fail and MP_JOIN will fail too, incrementing MPTCP_MIB_JOINNOTOKEN.

I'm looking at the above issue.

I think we can address it allocating earlier the msk socket in subflow_syn_recv_sock(), so that we can always insert the 'token -> msk' association into the radix tree before entering the fully established status.

As a nice side effect mptcp_connect() will be simpler and faster.

The flip side is more memory allocated for the child socket backlog (I think it should not be that bad, as the child backlog is limited).

I hope to share some early code soon. (will be a separate series anyway)

Any comments/opinion/warns ?!?

Thanks,

Paolo

#5 lore raw

[MPTCP] Re: [PATCH v1 4/0004] Squash-to: "mptcp: Add handling of outgoing MP_JOIN requests"

In reply to Paolo Abeni
Quoted 2 lines · Paolo Abeni

Schedule 3rd ack retransmission via delack timer, and clear it on fully established event

It seems ok to me to borrow the delack timer. Is it enough to only retransmit the 3rd ack one time?

Quoted 50 lines
Signed-off-by Paolo Abeni <pabeni(a)redhat.com>

---
net/mptcp/options.c | 61 ++++++++++++++++++++++++++++++++++++++-------

1 file changed, 52 insertions(+), 9 deletions(-)
net/mptcp/options.c +32 -0
index eb688b3f759f..6f01ae8efeee 100644
@@ -341,6 +341,38 @@ void mptcp_rcv_synsent(struct sock *sk)
 	}
 }
 
+/* MP_JOIN client subflow must wait for 4th ack before sending any data:
+ * TCP can't schedule delack timer before the subflow is fully established.
+ * MPTCP uses the delack timer to do 3rd ack retransmissions
+ */
+static void schedule_3rdack_retransmission(struct sock *sk)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+	struct tcp_sock *tp = tcp_sk(sk);
+	unsigned long timeout;
+
+	/* reschedule with a timeout above RTT, as we must look only for drop */
+	if (tp->srtt_us)
+		timeout = tp->srtt_us << 1;
+	else
+		timeout = TCP_TIMEOUT_INIT;
+
+	WARN_ON_ONCE(icsk->icsk_ack.pending & ICSK_ACK_TIMER);
+	icsk->icsk_ack.pending |= ICSK_ACK_SCHED | ICSK_ACK_TIMER;
+	icsk->icsk_ack.timeout = timeout;
+	sk_reset_timer(sk, &icsk->icsk_delack_timer, timeout);
+}
+
+static void clear_3rdack_retransmission(struct sock *sk)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+
+	sk_stop_timer(sk, &icsk->icsk_delack_timer);
+	icsk->icsk_ack.timeout = 0;
+	icsk->icsk_ack.ato = 0;
+	icsk->icsk_ack.pending &= ~(ICSK_ACK_SCHED | ICSK_ACK_TIMER);
+}
+
 static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
 					 unsigned int *size,
 					 unsigned int remaining,

net/mptcp/options.c static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb, · line 388

Quoted 10 lines
net/mptcp/options.c +4 -3
@@ -350,12 +382,20 @@ static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
 	struct mptcp_ext *mpext;
 	unsigned int data_len;
 
-	pr_debug("subflow=%p fully established=%d seq=%x:%x remaining=%d",
-		 subflow, subflow->fully_established, subflow->snd_isn,
-		 skb ? TCP_SKB_CB(skb)->seq : 0, remaining);
+	/* When skb is not available, we better over-estimate the emitted
+	 * options len. A full DSS option (28 bytes) is longer than
+	 * TCPOLEN_MPTCP_MPC_ACK_DATA(22) or TCPOLEN_MPTCP_MPJ_ACK(24), so
+	 * let's the caller try to fit that.

Since *size is not set within this function, I suggest replacing this last line with "tell the caller to defer the estimate to mptcp_established_options_dss(), which will reserve enough space".

net/mptcp/options.c static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb, · line 395

Quoted 7 lines
net/mptcp/options.c +7 -0
+	 */
+	if (!skb)
+		return false;
+
+	/* MPC/MPJ needed only on 3rd ack packet */
+	if (subflow->snd_isn != TCP_SKB_CB(skb)->seq ||
+	    subflow->fully_established)

snd_isn is going to match sometimes when the sequence numbers wrap around, I think it would be good to check subflow->fully_established first to clarify that snd_isn is only meaningful in the !subflow_fully_established case.

net/mptcp/options.c static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb, · line 401

Quoted 8 lines
net/mptcp/options.c +2 -2
+		return false;
 
-	if (subflow->mp_capable && !subflow->fully_established && skb &&
-	    subflow->snd_isn == TCP_SKB_CB(skb)->seq) {
+	if (subflow->mp_capable) {
 		/* When skb is not available, we better over-estimate the
 		 * emitted options len. A full DSS option is longer than
 		 * TCPOLEN_MPTCP_MPC_ACK_DATA, so let's the caller try to fit

Looks like this comment was left in when copy/pasting

Quoted 33 lines
net/mptcp/options.c +7 -4
@@ -388,12 +428,13 @@ static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
 			 data_len);
 
 		return true;
-	} else if (subflow->mp_join && !subflow->fourth_ack) {
+	} else if (subflow->mp_join) {
 		opts->suboptions = OPTION_MPTCP_MPJ_ACK;
 		memcpy(opts->hmac, subflow->hmac, MPTCPOPT_HMAC_LEN);
 		*size = TCPOLEN_MPTCP_MPJ_ACK;
-		subflow->fourth_ack = 1;
 		pr_debug("subflow=%p", subflow);
+
+		schedule_3rdack_retransmission(sk);
 		return true;
 	}
 	return false;
@@ -633,10 +674,12 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *sk,
 		return true;
 
 	subflow->pm_notified = 1;
-	if (subflow->mp_join)
+	if (subflow->mp_join) {
+		clear_3rdack_retransmission(sk);
 		mptcp_pm_subflow_established(msk, subflow);
-	else
+	} else {
 		mptcp_pm_fully_established(msk);
+	}
 	return true;
 }
 
Signature · 1 line
2.21.1

--
Mat Martineau
Intel

#16 lore raw
In reply to Paolo Abeni
Quoted 17 lines · Paolo Abeni
Quoted 6 lines · Paolo Abeni

Note: mp join self tests are still unstable, e.g. can fail randomly, due to an old race between mptcp_token_update_accept() and mptcp_token_get_sock(): e.g. a client may legitly try to create a subflow after that the first one is fully established but before the server has accepted the initial socket:
mptcp_token_get_sock() will fail and MP_JOIN will fail too, incrementing MPTCP_MIB_JOINNOTOKEN.

I'm looking at the above issue.

I think we can address it allocating earlier the msk socket in subflow_syn_recv_sock(), so that we can always insert the 'token -> msk' association into the radix tree before entering the fully established status.

As a nice side effect mptcp_connect() will be simpler and faster.

Quoted 8 lines

The flip side is more memory allocated for the child socket backlog (I think it should not be that bad, as the child backlog is limited).

I hope to share some early code soon. (will be a separate series anyway)

Any comments/opinion/warns ?!?

Would it work to do the allocation in mptcp_stream_accept() instead of subflow_syn_recv_sock() to avoid the backlog issue? At that point you know the server is accepting the connection, and it looks like the msk lock isn't held yet so you still have some flexibility there.

Also - I did look through the full patch set and just had the comments on patch 4.

--
Mat Martineau
Intel

author
#6 lore raw

[MPTCP] Re: [PATCH v1 4/0004] Squash-to: "mptcp: Add handling of outgoing MP_JOIN requests"

In reply to Mat Martineau

Hi,

Thank you for the detailed review!

Quoted 7 lines · Mat Martineau
Quoted 2 lines · Paolo Abeni

Schedule 3rd ack retransmission via delack timer, and clear it on fully established event

It seems ok to me to borrow the delack timer. Is it enough to only retransmit the 3rd ack one time?

The delack timer is fired every time we transmit the 3rd ack, so possibly more then once/as many time as needed - modulo bugs ;)

[...]

Quoted 7 lines
Quoted 7 lines

static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
unsigned int *size, unsigned int remaining,

net/mptcp/options.c +0 -0
@@ -350,12 +382,20 @@ static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
 	struct mptcp_ext *mpext;
 	unsigned int data_len;
 
Quoted 11 lines

net/mptcp/options.c static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb, · line 388

Quoted 7 lines
net/mptcp/options.c +4 -3
-	pr_debug("subflow=%p fully established=%d seq=%x:%x remaining=%d",
-		 subflow, subflow->fully_established, subflow->snd_isn,
-		 skb ? TCP_SKB_CB(skb)->seq : 0, remaining);
+	/* When skb is not available, we better over-estimate the emitted
+	 * options len. A full DSS option (28 bytes) is longer than
+	 * TCPOLEN_MPTCP_MPC_ACK_DATA(22) or TCPOLEN_MPTCP_MPJ_ACK(24), so
+	 * let's the caller try to fit that.

Since *size is not set within this function, I suggest replacing this last line with "tell the caller to defer the estimate to mptcp_established_options_dss(), which will reserve enough space".

Ok, will do in the next iteration.

Quoted 4 lines
Quoted 4 lines
net/mptcp/options.c +4 -0
+	 */
+	if (!skb)
+		return false;
+
Quoted 8 lines

net/mptcp/options.c static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb, · line 395

Quoted 3 lines
net/mptcp/options.c +3 -0
+	/* MPC/MPJ needed only on 3rd ack packet */
+	if (subflow->snd_isn != TCP_SKB_CB(skb)->seq ||
+	    subflow->fully_established)

snd_isn is going to match sometimes when the sequence numbers wrap around, I think it would be good to check subflow->fully_established first to clarify that snd_isn is only meaningful in the !subflow_fully_established case.

Agreed, will do in the next iteration.

Quoted 10 lines

net/mptcp/options.c static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb, · line 401

Quoted 8 lines
net/mptcp/options.c +2 -2
+		return false;
 
-	if (subflow->mp_capable && !subflow->fully_established && skb &&
-	    subflow->snd_isn == TCP_SKB_CB(skb)->seq) {
+	if (subflow->mp_capable) {
 		/* When skb is not available, we better over-estimate the
 		 * emitted options len. A full DSS option is longer than
 		 * TCPOLEN_MPTCP_MPC_ACK_DATA, so let's the caller try to fit

Looks like this comment was left in when copy/pasting

Indeed. I'll drop it in the next iteration.

Cheers,

Paolo

author
#17 lore raw
In reply to Mat Martineau
Quoted 10 lines · Mat Martineau
Quoted 8 lines · Paolo Abeni
Quoted 6 lines · Paolo Abeni

Note: mp join self tests are still unstable, e.g. can fail randomly, due to an old race between mptcp_token_update_accept() and mptcp_token_get_sock(): e.g. a client may legitly try to create a subflow after that the first one is fully established but before the server has accepted the initial socket:
mptcp_token_get_sock() will fail and MP_JOIN will fail too, incrementing MPTCP_MIB_JOINNOTOKEN.

Quoted 8 lines
Quoted 8 lines

I'm looking at the above issue.

I think we can address it allocating earlier the msk socket in subflow_syn_recv_sock(), so that we can always insert the 'token -> msk' association into the radix tree before entering the fully established status.

As a nice side effect mptcp_connect() will be simpler and faster.

typo here!!! s/mptcp_connect()/mptcp_accept()/, sorry.

Quoted 4 lines
Quoted 4 lines

The flip side is more memory allocated for the child socket backlog (I think it should not be that bad, as the child backlog is limited).

Quoted 8 lines
Quoted 5 lines

I hope to share some early code soon. (will be a separate series anyway)

Any comments/opinion/warns ?!?

Would it work to do the allocation in mptcp_stream_accept() instead of subflow_syn_recv_sock() to avoid the backlog issue?

That is more or less what we are doing now (allocation happens in mptcp_accept()), so it would not help.

As per the addittional memory usage, I think that should be fine because we are limited by the 'backlog' argument in the listen() syscall, and very large backlog values will use a lot of memory no matter what.

Quoted 2 lines

Also - I did look through the full patch set and just had the comments on patch 4.

Thank you!

/Paolo