Skip to contentSkip to conversation

add add_addr_urgent

PATCH Superseded Geliang Tang 3 participants 9 messages Jan 6, 2021 – Jan 19, 2021

Subsystems (4)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Geliang Tang

author
#0 lore raw

cover add add_addr_urgent

This patchset addresses the ADD_ADDR timeout issue, the failure output looks like this:

12 signal address, ADD_ADDR timeout syn[ ok ] - synack[ ok ] - ack[ ok ]
add[fail] got 2 ADD_ADDR[s] expected 4
- echo [ ok ]
Server ns stats

 MPTcpExtMPCapableSYNRX          1               0.0
 MPTcpExtMPCapableACKRX          1               0.0
 MPTcpExtMPJoinSynRx             1               0.0
 MPTcpExtMPJoinAckRx             1               0.0
 Client ns stats
 MPTcpExtMPJoinSynAckRx          1               0.0
 MPTcpExtAddAddr                 2               0.0

13 remove single subflow syn[ ok ] - synack[ ok ] - ack[ ok ]
rm [ ok ] - sf [ ok ]

The tasecase needs to send out 4 ADD_ADDR packets in the test process, it's about 22 round trips when we use 'slow' argument in the testcase.

If we can't send out all the 4 ADD_ADDR packets, the testcase will fail. Here in the failure case, we only sent out 2 ADD_ADDRs.

But the ADD_ADDR packet cannot sent out immediately, we need to find out a packet with enough option length space to put in the ADD_ADDR suboption. So we need to try several times before we can find a suitable packet to sent out a ADD_ADDR suboption.

I added the following test code to count the ADD_ADDR trying times:

---
@@ -233,6 +233,7 @@ void mptcp_pm_mp_prio_received(struct sock *sk, u8 bkup)

/* path manager helpers */

+static int times = 1;
bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int remaining,
struct mptcp_addr_info *saddr, bool *echo, bool *port)
{
@@ -247,11 +248,14 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int remaining,

         *echo = mptcp_pm_should_add_signal_echo(msk);
         *port = mptcp_pm_should_add_signal_port(msk);

- if (remaining < mptcp_add_addr_len(msk->pm.local.family, *echo, *port))
+ if (remaining < mptcp_add_addr_len(msk->pm.local.family, *echo, *port)) {

 +               pr_debug("%s remaining=%u times=%d\n", __func__, remaining, times++);
                 goto out_unlock;

+ }

         *saddr = msk->pm.local;
         WRITE_ONCE(msk->pm.addr_signal, 0);
 +       times = 1;
         ret = true;

out_unlock:
---

Then I found that when the ADD_ADDR timeout testcase failed, it had tried about 20 times to send out the ADD_ADDR. And here the test transmit process ended, so the last two ADD_ADDRs had no chance to be send out.

We have three solutions to fix this issue:

1. Make the test transmit process more slowly.
2. Make the transmit data biger.
3. Make the ADD_ADDR suboption transmit faster.

The first 2 solutions can partly solve the issue, it makes more round trips in the tranmit process, and all 4 ADD_ADDRs had more chance to be send out. But in some cases, the issue is still there.

So I chose the last sulution to fix the issue. This patchset adds the urgent status for ADD_ADDR, to make sure the ADD_ADDR can be sent out immediately.

Geliang Tang (5):
mptcp: add mptcp_pm_should_add_signal_send_ack helper
mptcp: simplify mptcp_pm_nl_add_addr_send_ack debug log
mptcp: add a new sysctl add_addr_urgent
mptcp: add urgent status for ADD_ADDR
selftests: mptcp: add add_addr_urgent testcases

 Documentation/networking/mptcp-sysctl.rst     |  8 ++++++
 net/mptcp/ctrl.c                              | 14 ++++++++++
 net/mptcp/options.c                           |  3 +-
 net/mptcp/pm.c                                |  5 ++--
 net/mptcp/pm_netlink.c                        | 13 ++++++---
 net/mptcp/protocol.h                          | 14 ++++++++++
 .../testing/selftests/net/mptcp/mptcp_join.sh | 28 +++++++++++++++++++
 7 files changed, 77 insertions(+), 8 deletions(-)
Signature · 1 line
2.29.2
author
#1 lore raw

PATCH 1/5 mptcp: add mptcp_pm_should_add_signal_send_ack helper

This patch added a new helper function named mptcp_pm_should_add_signal_send_ack, in it put together all the mptcp_addr_signal_status which need to send out a dedicated ADD_ADDR packet.

Signed-off-by Geliang Tang <geliangtang(a)gmail.com>
---
 net/mptcp/options.c    | 3 +--
 net/mptcp/pm.c         | 3 +--
 net/mptcp/pm_netlink.c | 3 +--
 net/mptcp/protocol.h   | 6 ++++++
 4 files changed, 9 insertions(+), 6 deletions(-)
net/mptcp/options.c +1 -2
index 331d460d82fa..0c55795a8692 100644
@@ -611,8 +611,7 @@ static bool mptcp_established_options_add_addr(struct sock *sk, struct sk_buff *
 	bool port;
 	int len;
 
-	if ((mptcp_pm_should_add_signal_ipv6(msk) ||
-	     mptcp_pm_should_add_signal_port(msk)) &&
+	if (mptcp_pm_should_add_signal_send_ack(msk) &&
 	    skb && skb_is_tcp_pure_ack(skb)) {
 		pr_debug("drop other suboptions");
 		opts->suboptions = 0;
net/mptcp/pm.c +1 -2
index 01a846b25771..c22a43b20323 100644
@@ -191,8 +191,7 @@ void mptcp_pm_add_addr_received(struct mptcp_sock *msk,
 
 void mptcp_pm_add_addr_send_ack(struct mptcp_sock *msk)
 {
-	if (!mptcp_pm_should_add_signal_ipv6(msk) &&
-	    !mptcp_pm_should_add_signal_port(msk))
+	if (!mptcp_pm_should_add_signal_send_ack(msk))
 		return;
 
 	mptcp_pm_schedule_work(msk, MPTCP_PM_ADD_ADDR_SEND_ACK);
net/mptcp/pm_netlink.c +1 -2
index f736bec184c5..9a8a5d9c1633 100644
@@ -494,8 +494,7 @@ void mptcp_pm_nl_add_addr_send_ack(struct mptcp_sock *msk)
 {
 	struct mptcp_subflow_context *subflow;
 
-	if (!mptcp_pm_should_add_signal_ipv6(msk) &&
-	    !mptcp_pm_should_add_signal_port(msk))
+	if (!mptcp_pm_should_add_signal_send_ack(msk))
 		return;
 
 	__mptcp_flush_join_list(msk);
net/mptcp/protocol.h +6 -0
index eba1c83cc090..964bf66f7221 100644
@@ -598,6 +598,12 @@ static inline bool mptcp_pm_should_rm_signal(struct mptcp_sock *msk)
 	return READ_ONCE(msk->pm.addr_signal) & BIT(MPTCP_RM_ADDR_SIGNAL);
 }
 
+static inline bool mptcp_pm_should_add_signal_send_ack(struct mptcp_sock *msk)
+{
+	return mptcp_pm_should_add_signal_ipv6(msk) ||
+	       mptcp_pm_should_add_signal_port(msk);
+}
+
 static inline unsigned int mptcp_add_addr_len(int family, bool echo, bool port)
 {
 	u8 len = TCPOLEN_MPTCP_ADD_ADDR_BASE;
Signature · 1 line
2.29.2
author
#2 lore raw

PATCH 2/5 mptcp: simplify mptcp_pm_nl_add_addr_send_ack debug log

Print out the debug log of mptcp_pm_nl_add_addr_send_ack in a single line when more than one mptcp_addr_signal_status are set.

Signed-off-by Geliang Tang <geliangtang(a)gmail.com>
---
 net/mptcp/pm_netlink.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
net/mptcp/pm_netlink.c +4 -2
index 9a8a5d9c1633..06c9ec12c538 100644
@@ -501,13 +501,15 @@ void mptcp_pm_nl_add_addr_send_ack(struct mptcp_sock *msk)
 	subflow = list_first_entry_or_null(&msk->conn_list, typeof(*subflow), node);
 	if (subflow) {
 		struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+		char buf[32] = { 0 };
 		u8 add_addr;
 
 		spin_unlock_bh(&msk->pm.lock);
 		if (mptcp_pm_should_add_signal_ipv6(msk))
-			pr_debug("send ack for add_addr6");
+			strcat(buf, "[ipv6]");
 		if (mptcp_pm_should_add_signal_port(msk))
-			pr_debug("send ack for add_addr_port");
+			strcat(buf, "[port]");
+		pr_debug("send ack for add_addr %s", buf);
 
 		lock_sock(ssk);
 		tcp_send_ack(ssk);
Signature · 1 line
2.29.2
author
#3 lore raw

PATCH 3/5 mptcp: add a new sysctl add_addr_urgent

This patch added a new sysctl named add_addr_urgent, to control whether the ADD_ADDR suboption can be sent out immediately in only one round trip.

Signed-off-by Geliang Tang <geliangtang(a)gmail.com>
---
 Documentation/networking/mptcp-sysctl.rst |  8 ++++++++
 net/mptcp/ctrl.c                          | 14 ++++++++++++++
 net/mptcp/protocol.h                      |  1 +
 3 files changed, 23 insertions(+)
Documentation/networking/mptcp-sysctl.rst +8 -0
index 6af0196c4297..d7984701e411 100644
@@ -24,3 +24,11 @@ add_addr_timeout - INTEGER (seconds)
 	sysctl.
 
 	Default: 120
+
+add_addr_urgent - INTEGER
+	Control whether the ADD_ADDR suboption can be sent out immediately
+	in only one round trip.
+
+	The default value is zero. This is a per-namespace sysctl.
+
+	Default: 0
net/mptcp/ctrl.c +14 -0
index 96ba616f59bf..b0af5ff52136 100644
@@ -19,6 +19,7 @@ struct mptcp_pernet {
 
 	int mptcp_enabled;
 	unsigned int add_addr_timeout;
+	int add_addr_urgent;
 };
 
 static struct mptcp_pernet *mptcp_get_pernet(struct net *net)
@@ -36,6 +37,11 @@ unsigned int mptcp_get_add_addr_timeout(struct net *net)
 	return mptcp_get_pernet(net)->add_addr_timeout;
 }
 
+bool mptcp_get_add_addr_urgent(struct net *net)
+{
+	return mptcp_get_pernet(net)->add_addr_urgent;
+}
+
 static struct ctl_table mptcp_sysctl_table[] = {
 	{
 		.procname = "enabled",
@@ -52,6 +58,12 @@ static struct ctl_table mptcp_sysctl_table[] = {
 		.mode = 0644,
 		.proc_handler = proc_dointvec_jiffies,
 	},
+	{
+		.procname = "add_addr_urgent",
+		.maxlen = sizeof(int),
+		.mode = 0644,
+		.proc_handler = proc_dointvec,
+	},
 	{}
 };
 
@@ -59,6 +71,7 @@ static void mptcp_pernet_set_defaults(struct mptcp_pernet *pernet)
 {
 	pernet->mptcp_enabled = 1;
 	pernet->add_addr_timeout = TCP_RTO_MAX;
+	pernet->add_addr_urgent = 0;
 }
 
 static int mptcp_pernet_new_table(struct net *net, struct mptcp_pernet *pernet)
@@ -75,6 +88,7 @@ static int mptcp_pernet_new_table(struct net *net, struct mptcp_pernet *pernet)
 
 	table[0].data = &pernet->mptcp_enabled;
 	table[1].data = &pernet->add_addr_timeout;
+	table[2].data = &pernet->add_addr_urgent;
 
 	hdr = register_net_sysctl(net, MPTCP_SYSCTL_PATH, table);
 	if (!hdr)
net/mptcp/protocol.h +1 -0
index 964bf66f7221..56e200efec01 100644
@@ -461,6 +461,7 @@ static inline void mptcp_add_pending_subflow(struct mptcp_sock *msk,
 
 int mptcp_is_enabled(struct net *net);
 unsigned int mptcp_get_add_addr_timeout(struct net *net);
+bool mptcp_get_add_addr_urgent(struct net *net);
 void mptcp_subflow_fully_established(struct mptcp_subflow_context *subflow,
 				     struct mptcp_options_received *mp_opt);
 bool mptcp_subflow_data_available(struct sock *sk);
Signature · 1 line
2.29.2
author
#4 lore raw

PATCH 4/5 mptcp: add urgent status for ADD_ADDR

This patch added a new status named MPTCP_ADD_ADDR_URGENT in mptcp_addr_signal_status. If this status is set, a dedicated ADD_ADDR packet also needs to be sent out.

Signed-off-by Geliang Tang <geliangtang(a)gmail.com>
---
 net/mptcp/pm.c         | 2 ++
 net/mptcp/pm_netlink.c | 4 ++++
 net/mptcp/protocol.h   | 9 ++++++++-
 3 files changed, 14 insertions(+), 1 deletion(-)
net/mptcp/pm.c +2 -0
index c22a43b20323..998828990029 100644
@@ -33,6 +33,8 @@ int mptcp_pm_announce_addr(struct mptcp_sock *msk,
 		add_addr |= BIT(MPTCP_ADD_ADDR_IPV6);
 	if (port)
 		add_addr |= BIT(MPTCP_ADD_ADDR_PORT);
+	if (mptcp_get_add_addr_urgent(sock_net((struct sock *)msk)))
+		add_addr |= BIT(MPTCP_ADD_ADDR_URGENT);
 	WRITE_ONCE(msk->pm.addr_signal, add_addr);
 	return 0;
 }
net/mptcp/pm_netlink.c +4 -0
index 06c9ec12c538..01cd50646b82 100644
@@ -509,6 +509,8 @@ void mptcp_pm_nl_add_addr_send_ack(struct mptcp_sock *msk)
 			strcat(buf, "[ipv6]");
 		if (mptcp_pm_should_add_signal_port(msk))
 			strcat(buf, "[port]");
+		if (mptcp_pm_should_add_signal_urgent(msk))
+			strcat(buf, "[urgent]");
 		pr_debug("send ack for add_addr %s", buf);
 
 		lock_sock(ssk);
@@ -521,6 +523,8 @@ void mptcp_pm_nl_add_addr_send_ack(struct mptcp_sock *msk)
 			add_addr &= ~BIT(MPTCP_ADD_ADDR_IPV6);
 		if (mptcp_pm_should_add_signal_port(msk))
 			add_addr &= ~BIT(MPTCP_ADD_ADDR_PORT);
+		if (mptcp_pm_should_add_signal_urgent(msk))
+			add_addr &= ~BIT(MPTCP_ADD_ADDR_URGENT);
 		WRITE_ONCE(msk->pm.addr_signal, add_addr);
 	}
 }
net/mptcp/protocol.h +8 -1
index 56e200efec01..fd1c785bdcdf 100644
@@ -183,6 +183,7 @@ enum mptcp_addr_signal_status {
 	MPTCP_ADD_ADDR_ECHO,
 	MPTCP_ADD_ADDR_IPV6,
 	MPTCP_ADD_ADDR_PORT,
+	MPTCP_ADD_ADDR_URGENT,
 	MPTCP_RM_ADDR_SIGNAL,
 };
 
@@ -594,6 +595,11 @@ static inline bool mptcp_pm_should_add_signal_port(struct mptcp_sock *msk)
 	return READ_ONCE(msk->pm.addr_signal) & BIT(MPTCP_ADD_ADDR_PORT);
 }
 
+static inline bool mptcp_pm_should_add_signal_urgent(struct mptcp_sock *msk)
+{
+	return READ_ONCE(msk->pm.addr_signal) & BIT(MPTCP_ADD_ADDR_URGENT);
+}
+
 static inline bool mptcp_pm_should_rm_signal(struct mptcp_sock *msk)
 {
 	return READ_ONCE(msk->pm.addr_signal) & BIT(MPTCP_RM_ADDR_SIGNAL);
@@ -602,7 +608,8 @@ static inline bool mptcp_pm_should_rm_signal(struct mptcp_sock *msk)
 static inline bool mptcp_pm_should_add_signal_send_ack(struct mptcp_sock *msk)
 {
 	return mptcp_pm_should_add_signal_ipv6(msk) ||
-	       mptcp_pm_should_add_signal_port(msk);
+	       mptcp_pm_should_add_signal_port(msk) ||
+	       mptcp_pm_should_add_signal_urgent(msk);
 }
 
 static inline unsigned int mptcp_add_addr_len(int family, bool echo, bool port)
Signature · 1 line
2.29.2
author
#5 lore raw

PATCH 5/5 selftests: mptcp: add add_addr_urgent testcases

This patch added the multiple addresses testcases for add_addr_urgent in signal_address_tests and add_tests.

It also set add_addr_urgent sysctl in add_addr_timeout testcase to fix the ADD_ADDR timeout issue like this:

12 signal address, ADD_ADDR timeout syn[ ok ] - synack[ ok ] - ack[ ok ]
add[fail] got 2 ADD_ADDR[s] expected 4
- echo [ ok ]
Server ns stats

 MPTcpExtMPCapableSYNRX          1              0.0
 MPTcpExtMPCapableACKRX          1              0.0
 MPTcpExtMPJoinSynRx             1              0.0
 MPTcpExtMPJoinAckRx             1              0.0
 Client ns stats
 MPTcpExtMPJoinSynAckRx          1              0.0
 MPTcpExtAddAddr                 2              0.0

13 remove single subflow syn[ ok ] - synack[ ok ] - ack[ ok ]
rm [ ok ] - sf [ ok ]

Signed-off-by Geliang Tang <geliangtang(a)gmail.com>
---
 .../testing/selftests/net/mptcp/mptcp_join.sh | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)
tools/testing/selftests/net/mptcp/mptcp_join.sh +28 -0
index 540d98243a1e..1035f2d9f841 100755
@@ -114,6 +114,7 @@ reset_with_add_addr_timeout()
 	reset
 
 	ip netns exec $ns1 sysctl -q net.mptcp.add_addr_timeout=1
+	ip netns exec $ns1 sysctl -q net.mptcp.add_addr_urgent=1
 	ip netns exec $ns2 $tables -A OUTPUT -p tcp \
 		-m tcp --tcp-option 30 \
 		-m bpf --bytecode \
@@ -121,6 +122,14 @@ reset_with_add_addr_timeout()
 		-j DROP
 }
 
+reset_with_add_addr_urgent()
+{
+	reset
+
+	ip netns exec $ns1 sysctl -q net.mptcp.add_addr_urgent=1
+	ip netns exec $ns2 sysctl -q net.mptcp.add_addr_urgent=1
+}
+
 for arg in "$@"; do
 	if [ "$arg" = "-c" ]; then
 		capture=1
@@ -767,6 +776,17 @@ signal_address_tests()
 	run_tests $ns1 $ns2 10.0.1.1
 	chk_join_nr "multiple subflows and signal" 3 3 3
 	chk_add_nr 1 1
+
+	# signal multiple addresses, urgent
+	reset_with_add_addr_urgent
+	ip netns exec $ns1 ./pm_nl_ctl limits 0 3
+	ip netns exec $ns2 ./pm_nl_ctl limits 3 3
+	ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal
+	ip netns exec $ns1 ./pm_nl_ctl add 10.0.3.1 flags signal
+	ip netns exec $ns1 ./pm_nl_ctl add 10.0.4.1 flags signal
+	run_tests $ns1 $ns2 10.0.1.1
+	chk_join_nr "signal multiple addresses, urgent" 3 3 3
+	chk_add_nr 3 3
 }
 
 link_failure_tests()
@@ -895,6 +915,14 @@ add_tests()
 	run_tests $ns1 $ns2 10.0.1.1 0 0 2 slow
 	chk_join_nr "add multiple subflows" 2 2 2
 
+	# add multiple addresses
+	reset_with_add_addr_urgent
+	ip netns exec $ns1 ./pm_nl_ctl limits 0 2
+	ip netns exec $ns2 ./pm_nl_ctl limits 2 2
+	run_tests $ns1 $ns2 10.0.1.1 0 2 0 slow
+	chk_join_nr "add multiple addresses" 2 2 2
+	chk_add_nr 2 2
+
 	# add multiple subflows IPv6
 	reset
 	ip netns exec $ns1 ./pm_nl_ctl limits 0 2
Signature · 1 line
2.29.2
#6 lore raw
In reply to Geliang Tang
Quoted 57 lines · Geliang Tang

This patchset addresses the ADD_ADDR timeout issue, the failure output looks like this:

12 signal address, ADD_ADDR timeout syn[ ok ] - synack[ ok ] - ack[ ok ]
add[fail] got 2 ADD_ADDR[s] expected 4
- echo [ ok ]
Server ns stats

MPTcpExtMPCapableSYNRX          1               0.0
MPTcpExtMPCapableACKRX          1               0.0
MPTcpExtMPJoinSynRx             1               0.0
MPTcpExtMPJoinAckRx             1               0.0
Client ns stats
MPTcpExtMPJoinSynAckRx          1               0.0
MPTcpExtAddAddr                 2               0.0

13 remove single subflow syn[ ok ] - synack[ ok ] - ack[ ok ]
rm [ ok ] - sf [ ok ]

The tasecase needs to send out 4 ADD_ADDR packets in the test process, it's about 22 round trips when we use 'slow' argument in the testcase.

If we can't send out all the 4 ADD_ADDR packets, the testcase will fail. Here in the failure case, we only sent out 2 ADD_ADDRs.

But the ADD_ADDR packet cannot sent out immediately, we need to find out a packet with enough option length space to put in the ADD_ADDR suboption. So we need to try several times before we can find a suitable packet to sent out a ADD_ADDR suboption.

I added the following test code to count the ADD_ADDR trying times:

---

(quoted diff) +5 -2
@@ -233,6 +233,7 @@ void mptcp_pm_mp_prio_received(struct sock *sk, u8 bkup)
 
 /* path manager helpers */
 
+static int times = 1;
 bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int remaining,
                              struct mptcp_addr_info *saddr, bool *echo, bool *port)
 {
@@ -247,11 +248,14 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int remaining,
        *echo = mptcp_pm_should_add_signal_echo(msk);
        *port = mptcp_pm_should_add_signal_port(msk);
 
-       if (remaining < mptcp_add_addr_len(msk->pm.local.family, *echo, *port))
+       if (remaining < mptcp_add_addr_len(msk->pm.local.family, *echo, *port)) {
+               pr_debug("%s remaining=%u times=%d\n", __func__, remaining, times++);
                goto out_unlock;
+       }
 
        *saddr = msk->pm.local;
        WRITE_ONCE(msk->pm.addr_signal, 0);
+       times = 1;
        ret = true;
 
 out_unlock:
---
Quoted 3 lines

Then I found that when the ADD_ADDR timeout testcase failed, it had tried about 20 times to send out the ADD_ADDR. And here the test transmit process ended, so the last two ADD_ADDRs had no chance to be send out.

In your tests, was it clear what other options were taking the space and preventing ADD_ADDR from being sent?

Quoted 6 lines

We have three solutions to fix this issue:

1. Make the test transmit process more slowly.
2. Make the transmit data biger.
3. Make the ADD_ADDR suboption transmit faster.

Quoted 6 lines

The first 2 solutions can partly solve the issue, it makes more round trips in the tranmit process, and all 4 ADD_ADDRs had more chance to be send out. But in some cases, the issue is still there.

So I chose the last sulution to fix the issue. This patchset adds the urgent status for ADD_ADDR, to make sure the ADD_ADDR can be sent out immediately.

ADD_ADDR would mostly be used on longer-lived connections (if I'm mistaken about this, please educate me!), especially adding multiple addresses. Given that, I don't think the special case code and a new sysctl are the way this should be solved to make a quick test work.

What if mptcp_pm_should_add_signal_send_ack() returned true whenever any ADD_ADDR was pending - not just when ipv6 addresses or port numbers are included?

Mat

Quoted 9 lines

Geliang Tang (5):
mptcp: add mptcp_pm_should_add_signal_send_ack helper
mptcp: simplify mptcp_pm_nl_add_addr_send_ack debug log
mptcp: add a new sysctl add_addr_urgent
mptcp: add urgent status for ADD_ADDR
selftests: mptcp: add add_addr_urgent testcases

Quoted 15 lines
Documentation/networking/mptcp-sysctl.rst     |  8 ++++++
net/mptcp/ctrl.c                              | 14 ++++++++++
net/mptcp/options.c                           |  3 +-
net/mptcp/pm.c                                |  5 ++--
net/mptcp/pm_netlink.c                        | 13 ++++++---
net/mptcp/protocol.h                          | 14 ++++++++++
.../testing/selftests/net/mptcp/mptcp_join.sh | 28 +++++++++++++++++++
7 files changed, 77 insertions(+), 8 deletions(-)
Signature · 4 lines
2.29.2
_______________________________________________
mptcp mailing list -- mptcp(a)lists.01.org
To unsubscribe send an email to mptcp-leave(a)lists.01.org

--
Mat Martineau
Intel

#7 lore raw
Quoted 80 lines · Mat Martineau
Quoted 57 lines · Geliang Tang

This patchset addresses the ADD_ADDR timeout issue, the failure output looks like this:

12 signal address, ADD_ADDR timeout syn[ ok ] - synack[ ok ] - ack[ ok ]
add[fail] got 2 ADD_ADDR[s] expected 4
- echo [ ok ]
Server ns stats

MPTcpExtMPCapableSYNRX          1               0.0
MPTcpExtMPCapableACKRX          1               0.0
MPTcpExtMPJoinSynRx             1               0.0
MPTcpExtMPJoinAckRx             1               0.0
Client ns stats
MPTcpExtMPJoinSynAckRx          1               0.0
MPTcpExtAddAddr                 2               0.0

13 remove single subflow syn[ ok ] - synack[ ok ] - ack[ ok ]
rm [ ok ] - sf [ ok ]

The tasecase needs to send out 4 ADD_ADDR packets in the test process, it's about 22 round trips when we use 'slow' argument in the testcase.

If we can't send out all the 4 ADD_ADDR packets, the testcase will fail. Here in the failure case, we only sent out 2 ADD_ADDRs.

But the ADD_ADDR packet cannot sent out immediately, we need to find out a packet with enough option length space to put in the ADD_ADDR suboption. So we need to try several times before we can find a suitable packet to sent out a ADD_ADDR suboption.

I added the following test code to count the ADD_ADDR trying times:

---

(quoted diff) +5 -2
@@ -233,6 +233,7 @@ void mptcp_pm_mp_prio_received(struct sock *sk, u8 bkup)
 
 /* path manager helpers */
 
+static int times = 1;
 bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int remaining,
                              struct mptcp_addr_info *saddr, bool *echo, bool *port)
 {
@@ -247,11 +248,14 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int remaining,
        *echo = mptcp_pm_should_add_signal_echo(msk);
        *port = mptcp_pm_should_add_signal_port(msk);
 
-       if (remaining < mptcp_add_addr_len(msk->pm.local.family, *echo, *port))
+       if (remaining < mptcp_add_addr_len(msk->pm.local.family, *echo, *port)) {
+               pr_debug("%s remaining=%u times=%d\n", __func__, remaining, times++);
                goto out_unlock;
+       }
 
        *saddr = msk->pm.local;
        WRITE_ONCE(msk->pm.addr_signal, 0);
+       times = 1;
        ret = true;
 
 out_unlock:
---
Quoted 3 lines

Then I found that when the ADD_ADDR timeout testcase failed, it had tried about 20 times to send out the ADD_ADDR. And here the test transmit process ended, so the last two ADD_ADDRs had no chance to be send out.

In your tests, was it clear what other options were taking the space and preventing ADD_ADDR from being sent?

Quoted 6 lines

We have three solutions to fix this issue:

1. Make the test transmit process more slowly.
2. Make the transmit data biger.
3. Make the ADD_ADDR suboption transmit faster.

Quoted 6 lines

The first 2 solutions can partly solve the issue, it makes more round trips in the tranmit process, and all 4 ADD_ADDRs had more chance to be send out. But in some cases, the issue is still there.

So I chose the last sulution to fix the issue. This patchset adds the urgent status for ADD_ADDR, to make sure the ADD_ADDR can be sent out immediately.

Quoted 8 lines

ADD_ADDR would mostly be used on longer-lived connections (if I'm mistaken about this, please educate me!), especially adding multiple addresses. Given that, I don't think the special case code and a new sysctl are the way this should be solved to make a quick test work.

What if mptcp_pm_should_add_signal_send_ack() returned true whenever any ADD_ADDR was pending - not just when ipv6 addresses or port numbers are included?

I'm sorry for late feedback.

I agree with Mat: I think it would be better avoding adding any special code here. I think dup acks would be acceptable, as ADD_ADDR will be not so frequent events.

So let mptcp_pm_should_add_signal_send_ack() return true looks a suitable solution to me. Since mptcp_pm_should_add_signal_send_ack() is invoked only with ADD_ADDR pending, the code could additional simplified even dropping such function call -> always go ahead and send the ack.

Cheers,

Paolo

author
#8 lore raw

Hi Mat, Paolo,

Quoted 61 lines · Paolo Abeni <pabeni(a)redhat.com> 于2021年1月18日周一 下午4:53写道:
Quoted 59 lines · Mat Martineau
Quoted 57 lines · Geliang Tang

This patchset addresses the ADD_ADDR timeout issue, the failure output looks like this:

12 signal address, ADD_ADDR timeout syn[ ok ] - synack[ ok ] - ack[ ok ]
add[fail] got 2 ADD_ADDR[s] expected 4
- echo [ ok ]
Server ns stats

MPTcpExtMPCapableSYNRX          1               0.0
MPTcpExtMPCapableACKRX          1               0.0
MPTcpExtMPJoinSynRx             1               0.0
MPTcpExtMPJoinAckRx             1               0.0
Client ns stats
MPTcpExtMPJoinSynAckRx          1               0.0
MPTcpExtAddAddr                 2               0.0

13 remove single subflow syn[ ok ] - synack[ ok ] - ack[ ok ]
rm [ ok ] - sf [ ok ]

The tasecase needs to send out 4 ADD_ADDR packets in the test process, it's about 22 round trips when we use 'slow' argument in the testcase.

If we can't send out all the 4 ADD_ADDR packets, the testcase will fail. Here in the failure case, we only sent out 2 ADD_ADDRs.

But the ADD_ADDR packet cannot sent out immediately, we need to find out a packet with enough option length space to put in the ADD_ADDR suboption. So we need to try several times before we can find a suitable packet to sent out a ADD_ADDR suboption.

I added the following test code to count the ADD_ADDR trying times:

---

(quoted diff) +5 -2
@@ -233,6 +233,7 @@ void mptcp_pm_mp_prio_received(struct sock *sk, u8 bkup)
 
 /* path manager helpers */
 
+static int times = 1;
 bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int remaining,
                              struct mptcp_addr_info *saddr, bool *echo, bool *port)
 {
@@ -247,11 +248,14 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int remaining,
        *echo = mptcp_pm_should_add_signal_echo(msk);
        *port = mptcp_pm_should_add_signal_port(msk);
 
-       if (remaining < mptcp_add_addr_len(msk->pm.local.family, *echo, *port))
+       if (remaining < mptcp_add_addr_len(msk->pm.local.family, *echo, *port)) {
+               pr_debug("%s remaining=%u times=%d\n", __func__, remaining, times++);
                goto out_unlock;
+       }
 
        *saddr = msk->pm.local;
        WRITE_ONCE(msk->pm.addr_signal, 0);
+       times = 1;
        ret = true;
 
 out_unlock:
---
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

Then I found that when the ADD_ADDR timeout testcase failed, it had tried about 20 times to send out the ADD_ADDR. And here the test transmit process ended, so the last two ADD_ADDRs had no chance to be send out.

In your tests, was it clear what other options were taking the space and preventing ADD_ADDR from being sent?

In my test, DSS is taking the space and preventing ADD_ADDR from being sent.

Quoted 29 lines
Quoted 14 lines
Quoted 7 lines

We have three solutions to fix this issue:

1. Make the test transmit process more slowly.
2. Make the transmit data biger.
3. Make the ADD_ADDR suboption transmit faster.

Quoted 6 lines

The first 2 solutions can partly solve the issue, it makes more round trips in the tranmit process, and all 4 ADD_ADDRs had more chance to be send out. But in some cases, the issue is still there.

So I chose the last sulution to fix the issue. This patchset adds the urgent status for ADD_ADDR, to make sure the ADD_ADDR can be sent out immediately.

Quoted 8 lines

ADD_ADDR would mostly be used on longer-lived connections (if I'm mistaken about this, please educate me!), especially adding multiple addresses. Given that, I don't think the special case code and a new sysctl are the way this should be solved to make a quick test work.

What if mptcp_pm_should_add_signal_send_ack() returned true whenever any ADD_ADDR was pending - not just when ipv6 addresses or port numbers are included?

I'm sorry for late feedback.

I agree with Mat: I think it would be better avoding adding any special code here. I think dup acks would be acceptable, as ADD_ADDR will be not so frequent events.

Quoted 6 lines

So let mptcp_pm_should_add_signal_send_ack() return true looks a suitable solution to me. Since mptcp_pm_should_add_signal_send_ack() is invoked only with ADD_ADDR pending, the code could additional simplified even dropping such function call -> always go ahead and send the ack.

Thanks for your advice, I'll improve this patchset this week.

-Geliang

Quoted 4 lines

Cheers,

Paolo