mptcp: Refactor ADD_ADDR/RM_ADDR handling
Subsystems (4)
cover mptcp: Refactor ADD_ADDR/RM_ADDR handling
This patch set changes the way MPTCP ADD_ADDR and RM_ADDR options are handled to improve the reliability of sending and updating address advertisements. The information used to populate outgoing advertisement option headers is now stored separately to avoid rare cases where a more recent request would overwrite something that had not been sent yet. While the peers would recover from this, it's better to avoid the problem in the first place.
Patch 1 moves an advertisement option check under a lock so the changes made in the next several patches will not introduce a race.
Patches 2-4 make sure ADD_ADDR, ADD_ADDR echo, and RM_ADDR options use separate flags and data.
Patch 5 removes some now-redundant flags.
Patch 6 adds a selftest that confirms the advertisement reliability improvements.
Yonglong Li (6):
mptcp: move drop_other_suboptions check under pm lock
mptcp: make MPTCP_ADD_ADDR_SIGNAL and MPTCP_ADD_ADDR_ECHO separate
mptcp: fix ADD_ADDR and RM_ADDR maybe flush addr_signal each other
mptcp: build ADD_ADDR/echo-ADD_ADDR option according pm.add_signal
mptcp: remove MPTCP_ADD_ADDR_IPV6 and MPTCP_ADD_ADDR_PORT
selftests: mptcp: add_addr and echo race test
net/mptcp/options.c | 28 ++++----- net/mptcp/pm.c | 58 +++++++++++++------ net/mptcp/pm_netlink.c | 10 ++-- net/mptcp/protocol.h | 24 ++++---- .../testing/selftests/net/mptcp/mptcp_join.sh | 15 +++++ 5 files changed, 83 insertions(+), 52 deletions(-)
base-commit: f6a4e0e8a00ff6fadb29f3646ccd33cc85195a38
Signature · 1 line
2.33.0
PATCH 1/6 mptcp: move drop_other_suboptions check under pm lock
From: Yonglong Li <[email protected]>
This patch moved the drop_other_suboptions check from mptcp_established_options_add_addr() into mptcp_pm_add_addr_signal(), do it under the PM lock to avoid the race between this check and mptcp_pm_add_addr_signal().
For this, added a new parameter for mptcp_pm_add_addr_signal() to get the drop_other_suboptions value. And drop the other suboptions after the option length check if drop_other_suboptions is true.
Additionally, always drop the other suboption for TCP pure ack:
that makes both the code simpler and the MPTCP behaviour more consistent.
--- net/mptcp/options.c | 28 ++++++++++++++-------------- net/mptcp/pm.c | 15 +++++++++++++-- net/mptcp/protocol.h | 6 ++++-- 3 files changed, 31 insertions(+), 18 deletions(-)
net/mptcp/options.c +14 -14
@@ -667,29 +667,29 @@ static bool mptcp_established_options_add_addr(struct sock *sk, struct sk_buff *bool port;int len; -if ((mptcp_pm_should_add_signal_ipv6 (msk) || -mptcp_pm_should_add_signal_port (msk) || -mptcp_pm_should_add_signal_echo (msk)) && - skb &&skb_is_tcp_pure_ack (skb)) { -pr_debug ("drop other suboptions" ); - opts->suboptions =0 ; - opts->ext_copy.use_ack =0 ; - opts->ext_copy.use_map =0 ; - remaining += opt_size; - drop_other_suboptions = true; - } - +/* add addr will strip the existing options, be sure to avoid breaking +* MPC/MPJ handshakes +*/ if (!mptcp_pm_should_add_signal (msk) || - !(mptcp_pm_add_addr_signal (msk, remaining, &opts->addr, &echo, &port))) + (opts->suboptions & (OPTION_MPTCP_MPJ_ACK |OPTION_MPTCP_MPC_ACK )) || + !mptcp_pm_add_addr_signal (msk, skb, opt_size, remaining, &opts->addr, + &echo, &port, &drop_other_suboptions))return false; +if (drop_other_suboptions) + remaining += opt_size; len =mptcp_add_addr_len (opts->addr.family, echo, port);if (remaining < len)return false; *size = len; -if (drop_other_suboptions) +if (drop_other_suboptions) { +pr_debug ("drop other suboptions" ); + opts->suboptions =0 ; + opts->ext_copy.use_ack =0 ; + opts->ext_copy.use_map =0 ; *size -= opt_size; + } opts->suboptions |=OPTION_MPTCP_ADD_ADDR ;if (!echo) { opts->ahmac = add_addr_generate_hmac(msk->local_key,
net/mptcp/pm.c +13 -2
@@ -251,8 +251,10 @@ void mptcp_pm_mp_prio_received(struct sock *sk, u8 bkup)/* path manager helpers */ -bool mptcp_pm_add_addr_signal (struct mptcp_sock *msk,unsigned int remaining, -struct mptcp_addr_info *saddr,bool *echo,bool *port) +bool mptcp_pm_add_addr_signal (struct mptcp_sock *msk,struct sk_buff *skb, +unsigned int opt_size,unsigned int remaining, +struct mptcp_addr_info *saddr,bool *echo, +bool *port,bool *drop_other_suboptions) {int ret = false;
@@ -262,6 +264,15 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int remaining,if (!mptcp_pm_should_add_signal (msk)) goto out_unlock; +/* always drop every other options for pure ack ADD_ADDR; this is a +* plain dup-ack from TCP perspective. The other MPTCP-relevant info, +* if any, will be carried by the 'original' TCP ack +*/ +if (skb &&skb_is_tcp_pure_ack (skb)) { + remaining += opt_size; + *drop_other_suboptions = true; + } + *echo =mptcp_pm_should_add_signal_echo (msk); *port =mptcp_pm_should_add_signal_port (msk);
net/mptcp/protocol.h +4 -2
@@ -794,8 +794,10 @@ static inline int mptcp_rm_addr_len(const struct mptcp_rm_list *rm_list)return TCPOLEN_MPTCP_RM_ADDR_BASE +roundup (rm_list->nr -1 ,4 ) +1 ; } -bool mptcp_pm_add_addr_signal (struct mptcp_sock *msk,unsigned int remaining, -struct mptcp_addr_info *saddr,bool *echo,bool *port); +bool mptcp_pm_add_addr_signal (struct mptcp_sock *msk,struct sk_buff *skb, +unsigned int opt_size,unsigned int remaining, +struct mptcp_addr_info *saddr,bool *echo, +bool *port,bool *drop_other_suboptions);bool mptcp_pm_rm_addr_signal (struct mptcp_sock *msk,unsigned int remaining,struct mptcp_rm_list *rm_list);int mptcp_pm_get_local_id (struct mptcp_sock *msk,struct sock_common *skc);
Signature · 1 line
2.33.0
PATCH 2/6 mptcp: make MPTCP_ADD_ADDR_SIGNAL and MPTCP_ADD_ADDR_ECHO separate
In reply to Mat MartineauFrom: Yonglong Li <[email protected]>
Use MPTCP_ADD_ADDR_SIGNAL only for the action of sending ADD_ADDR, and use MPTCP_ADD_ADDR_ECHO only for the action of sending ADD_ADDR echo.
Use msk->pm.local to save the announced ADD_ADDR address only, and reuse msk->pm.remote to save the announced ADD_ADDR_ECHO address.
To prepare for the next patch.
--- net/mptcp/pm.c | 16 ++++++++++------ net/mptcp/pm_netlink.c | 4 ++-- net/mptcp/protocol.h | 6 ++++++ 3 files changed, 18 insertions(+), 8 deletions(-)
net/mptcp/pm.c +10 -6
@@ -20,19 +20,23 @@ int mptcp_pm_announce_addr(struct mptcp_sock *msk, {u8 add_addr =READ_ONCE (msk->pm.addr_signal); -pr_debug ("msk=%p, local_id=%d" , msk, addr->id); +pr_debug ("msk=%p, local_id=%d, echo=%d" , msk, addr->id, echo);lockdep_assert_held (&msk->pm.lock); -if (add_addr) { -pr_warn ("addr_signal error, add_addr=%d" , add_addr); +if (add_addr & + (echo ?BIT (MPTCP_ADD_ADDR_ECHO ) :BIT (MPTCP_ADD_ADDR_SIGNAL ))) { +pr_warn ("addr_signal error, add_addr=%d, echo=%d" , add_addr, echo);return -EINVAL ; } - msk->pm.local = *addr; - add_addr |=BIT (MPTCP_ADD_ADDR_SIGNAL ); -if (echo) +if (echo) { + msk->pm.remote = *addr; add_addr |=BIT (MPTCP_ADD_ADDR_ECHO ); + }else { + msk->pm.local = *addr; + add_addr |=BIT (MPTCP_ADD_ADDR_SIGNAL ); + }if (addr->family ==AF_INET6 ) add_addr |=BIT (MPTCP_ADD_ADDR_IPV6 );if (addr->port)
net/mptcp/pm_netlink.c +2 -2
@@ -317,14 +317,14 @@ static void mptcp_pm_add_timer(struct timer_list *timer)if (!entry->addr.id)return ; -if (mptcp_pm_should_add_signal (msk)) { +if (mptcp_pm_should_add_signal_addr (msk)) {sk_reset_timer (sk, timer, jiffies +TCP_RTO_MAX /8 ); goto out; }spin_lock_bh (&msk->pm.lock); -if (!mptcp_pm_should_add_signal (msk)) { +if (!mptcp_pm_should_add_signal_addr (msk)) {pr_debug ("retransmit ADD_ADDR id=%d" , entry->addr.id);mptcp_pm_announce_addr (msk, &entry->addr, false);mptcp_pm_add_addr_send_ack (msk);
net/mptcp/protocol.h +6 -0
@@ -747,6 +747,12 @@ void mptcp_event_addr_announced(const struct mptcp_sock *msk, const struct mptcpvoid mptcp_event_addr_removed (const struct mptcp_sock *msk,u8 id);static inline bool mptcp_pm_should_add_signal (struct mptcp_sock *msk) +{ +return READ_ONCE (msk->pm.addr_signal) & + (BIT (MPTCP_ADD_ADDR_SIGNAL ) |BIT (MPTCP_ADD_ADDR_ECHO )); +} + +static inline bool mptcp_pm_should_add_signal_addr (struct mptcp_sock *msk) {return READ_ONCE (msk->pm.addr_signal) &BIT (MPTCP_ADD_ADDR_SIGNAL ); }
Signature · 1 line
2.33.0
PATCH 3/6 mptcp: fix ADD_ADDR and RM_ADDR maybe flush addr_signal each other
In reply to Mat MartineauFrom: Yonglong Li <[email protected]>
ADD_ADDR shares pm.addr_signal with RM_ADDR, so after RM_ADDR/ADD_ADDR has done, we should not clean ADD_ADDR/RM_ADDR's addr_signal.
--- net/mptcp/pm.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
net/mptcp/pm.c +10 -3
@@ -261,6 +261,7 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, struct sk_buff *skb, bool *port, bool *drop_other_suboptions) { int ret = false; +u8 add_addr;spin_lock_bh (&msk->pm.lock);
@@ -284,7 +285,11 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, struct sk_buff *skb, goto out_unlock; *saddr = msk->pm.local; -WRITE_ONCE (msk->pm.addr_signal,0 ); +if (*echo) + add_addr = msk->pm.addr_signal & ~BIT (MPTCP_ADD_ADDR_ECHO ); +else + add_addr = msk->pm.addr_signal & ~BIT (MPTCP_ADD_ADDR_SIGNAL ); +WRITE_ONCE (msk->pm.addr_signal, add_addr); ret = true; out_unlock:
@@ -296,6 +301,7 @@ bool mptcp_pm_rm_addr_signal(struct mptcp_sock *msk, unsigned int remaining, struct mptcp_rm_list *rm_list) {int ret = false, len; +u8 rm_addr;spin_lock_bh (&msk->pm.lock);
@@ -303,16 +309,17 @@ bool mptcp_pm_rm_addr_signal(struct mptcp_sock *msk, unsigned int remaining,if (!mptcp_pm_should_rm_signal (msk)) goto out_unlock; + rm_addr = msk->pm.addr_signal & ~BIT (MPTCP_RM_ADDR_SIGNAL ); len =mptcp_rm_addr_len (&msk->pm.rm_list_tx);if (len <0 ) { -WRITE_ONCE (msk->pm.addr_signal,0 ); +WRITE_ONCE (msk->pm.addr_signal, rm_addr); goto out_unlock; }if (remaining < len) goto out_unlock; *rm_list = msk->pm.rm_list_tx; -WRITE_ONCE (msk->pm.addr_signal,0 ); +WRITE_ONCE (msk->pm.addr_signal, rm_addr); ret = true; out_unlock:
Signature · 1 line
2.33.0
PATCH 4/6 mptcp: build ADD_ADDR/echo-ADD_ADDR option according pm.add_signal
In reply to Mat MartineauFrom: Yonglong Li <[email protected]>
According to the MPTCP_ADD_ADDR_SIGNAL or MPTCP_ADD_ADDR_ECHO flag, build the ADD_ADDR/ADD_ADDR_ECHO option.
In mptcp_pm_add_addr_signal(), use opts->addr to save the announced ADD_ADDR or ADD_ADDR_ECHO address.
--- net/mptcp/pm.c | 14 +++++++++----- net/mptcp/protocol.h | 2 +- 2 files changed, 10 insertions(+), 6 deletions(-)
net/mptcp/pm.c +9 -5
@@ -257,11 +257,12 @@ void mptcp_pm_mp_prio_received(struct sock *sk, u8 bkup)bool mptcp_pm_add_addr_signal (struct mptcp_sock *msk,struct sk_buff *skb,unsigned int opt_size,unsigned int remaining, -struct mptcp_addr_info *saddr,bool *echo, +struct mptcp_addr_info *addr,bool *echo,bool *port,bool *drop_other_suboptions) {int ret = false;u8 add_addr; +u8 family;spin_lock_bh (&msk->pm.lock);
@@ -281,14 +282,17 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, struct sk_buff *skb, *echo =mptcp_pm_should_add_signal_echo (msk); *port =mptcp_pm_should_add_signal_port (msk); -if (remaining <mptcp_add_addr_len (msk->pm.local.family, *echo, *port)) + family = *echo ? msk->pm.remote.family : msk->pm.local.family; +if (remaining <mptcp_add_addr_len (family, *echo, *port)) goto out_unlock; - *saddr = msk->pm.local; -if (*echo) +if (*echo) { + *addr = msk->pm.remote; add_addr = msk->pm.addr_signal & ~BIT (MPTCP_ADD_ADDR_ECHO ); -else + }else { + *addr = msk->pm.local; add_addr = msk->pm.addr_signal & ~BIT (MPTCP_ADD_ADDR_SIGNAL ); + }WRITE_ONCE (msk->pm.addr_signal, add_addr); ret = true;
net/mptcp/protocol.h +1 -1
@@ -802,7 +802,7 @@ static inline int mptcp_rm_addr_len(const struct mptcp_rm_list *rm_list)bool mptcp_pm_add_addr_signal (struct mptcp_sock *msk,struct sk_buff *skb,unsigned int opt_size,unsigned int remaining, -struct mptcp_addr_info *saddr,bool *echo, +struct mptcp_addr_info *addr,bool *echo,bool *port,bool *drop_other_suboptions);bool mptcp_pm_rm_addr_signal (struct mptcp_sock *msk,unsigned int remaining,struct mptcp_rm_list *rm_list);
Signature · 1 line
2.33.0
PATCH 5/6 mptcp: remove MPTCP_ADD_ADDR_IPV6 and MPTCP_ADD_ADDR_PORT
In reply to Mat MartineauFrom: Yonglong Li <[email protected]>
MPTCP_ADD_ADDR_IPV6 and MPTCP_ADD_ADDR_PORT are not necessary, we can get these info from pm.local or pm.remote.
Drop mptcp_pm_should_add_signal_ipv6 and mptcp_pm_should_add_signal_port too.
--- net/mptcp/pm.c | 6 +----- net/mptcp/pm_netlink.c | 6 ++---- net/mptcp/protocol.h | 12 ------------ 3 files changed, 3 insertions(+), 21 deletions(-)
net/mptcp/pm.c +1 -5
@@ -37,10 +37,6 @@ int mptcp_pm_announce_addr(struct mptcp_sock *msk, msk->pm.local = *addr; add_addr |=BIT (MPTCP_ADD_ADDR_SIGNAL ); } -if (addr->family ==AF_INET6 ) - add_addr |=BIT (MPTCP_ADD_ADDR_IPV6 ); -if (addr->port) - add_addr |=BIT (MPTCP_ADD_ADDR_PORT );WRITE_ONCE (msk->pm.addr_signal, add_addr);return 0 ; }
@@ -280,7 +276,7 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, struct sk_buff *skb, } *echo =mptcp_pm_should_add_signal_echo (msk); - *port =mptcp_pm_should_add_signal_port (msk); + *port = !!(*echo ? msk->pm.remote.port : msk->pm.local.port); family = *echo ? msk->pm.remote.family : msk->pm.local.family;if (remaining <mptcp_add_addr_len (family, *echo, *port))
net/mptcp/pm_netlink.c +2 -4
@@ -647,10 +647,8 @@ void mptcp_pm_nl_addr_send_ack(struct mptcp_sock *msk)bool slow;spin_unlock_bh (&msk->pm.lock); -pr_debug ("send ack for %s%s%s" , -mptcp_pm_should_add_signal (msk) ?"add_addr" :"rm_addr" , -mptcp_pm_should_add_signal_ipv6 (msk) ?" [ipv6]" :"" , -mptcp_pm_should_add_signal_port (msk) ?" [port]" :"" ); +pr_debug ("send ack for %s" , +mptcp_pm_should_add_signal (msk) ?"add_addr" :"rm_addr" ); slow =lock_sock_fast (ssk);tcp_send_ack (ssk);
net/mptcp/protocol.h +0 -12
@@ -178,8 +178,6 @@ enum mptcp_pm_status {enum mptcp_addr_signal_status {MPTCP_ADD_ADDR_SIGNAL ,MPTCP_ADD_ADDR_ECHO , -MPTCP_ADD_ADDR_IPV6 , -MPTCP_ADD_ADDR_PORT ,MPTCP_RM_ADDR_SIGNAL , };
@@ -762,16 +760,6 @@ static inline bool mptcp_pm_should_add_signal_echo(struct mptcp_sock *msk)return READ_ONCE (msk->pm.addr_signal) &BIT (MPTCP_ADD_ADDR_ECHO ); } -static inline bool mptcp_pm_should_add_signal_ipv6 (struct mptcp_sock *msk) -{ -return READ_ONCE (msk->pm.addr_signal) &BIT (MPTCP_ADD_ADDR_IPV6 ); -} - -static inline bool mptcp_pm_should_add_signal_port (struct mptcp_sock *msk) -{ -return READ_ONCE (msk->pm.addr_signal) &BIT (MPTCP_ADD_ADDR_PORT ); -} -static inline bool mptcp_pm_should_rm_signal (struct mptcp_sock *msk) {return READ_ONCE (msk->pm.addr_signal) &BIT (MPTCP_RM_ADDR_SIGNAL );
Signature · 1 line
2.33.0
PATCH 6/6 selftests: mptcp: add_addr and echo race test
In reply to Mat MartineauFrom: Yonglong Li <[email protected]>
This patch added an extra test for the singal_address_tests() to do the ADD_ADDR and ADD_ADDR_ECHO race test.
--- tools/testing/selftests/net/mptcp/mptcp_join.sh | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
tools/testing/selftests/net/mptcp/mptcp_join.sh +15 -0
@@ -1016,6 +1016,21 @@ signal_address_tests() run_tests $ns1 $ns2 10.0.1.1 chk_join_nr "signal invalid addresses" 1 1 1 chk_add_nr 3 3 + + # signal addresses race test + reset + ip netns exec $ns1 ./pm_nl_ctl limits 4 4 + ip netns exec $ns2 ./pm_nl_ctl limits 4 4 + ip netns exec $ns1 ./pm_nl_ctl add 10.0.1.1 flags signal + ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal + ip netns exec $ns1 ./pm_nl_ctl add 10.0.3.1 flags signal + ip netns exec $ns1 ./pm_nl_ctl add 10.0.4.1 flags signal + ip netns exec $ns2 ./pm_nl_ctl add 10.0.1.2 flags signal + ip netns exec $ns2 ./pm_nl_ctl add 10.0.2.2 flags signal + ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags signal + ip netns exec $ns2 ./pm_nl_ctl add 10.0.4.2 flags signal + run_tests $ns1 $ns2 10.0.1.1 + chk_add_nr 4 4 } link_failure_tests()
Signature · 1 line
2.33.0
This series was applied to netdev/net-next.git (refs/heads/master):
Hello:
This series was applied to netdev/net-next.git (refs/heads/master):
Quoted 9 lines · On Mon, 23 Aug 2021 18:05:38 -0700 you wrote:
This patch set changes the way MPTCP
ADD_ADDRandRM_ADDRoptions are handled to improve the reliability of sending and updating address advertisements. The information used to populate outgoing advertisement option headers is now stored separately to avoid rare cases where a more recent request would overwrite something that had not been sent yet. While the peers would recover from this, it's better to avoid the problem in the first place.[...]
Here is the summary with links:
- [net-next,1/6] mptcp: move drop_other_suboptions check under pm lock
https://git.kernel.org/netdev/net-next/c/1f5e9e2f5fd5 - [net-next,2/6] mptcp: make MPTCP_ADD_ADDR_SIGNAL and MPTCP_ADD_ADDR_ECHO separate https://git.kernel.org/netdev/net-next/c/18fc1a922e24 - [net-next,3/6] mptcp: fix ADD_ADDR and RM_ADDR maybe flush addr_signal each other https://git.kernel.org/netdev/net-next/c/119c022096f5 - [net-next,4/6] mptcp: build ADD_ADDR/echo-ADD_ADDR option according pm.add_signal https://git.kernel.org/netdev/net-next/c/f462a446384d - [net-next,5/6] mptcp: remove MPTCP_ADD_ADDR_IPV6 and MPTCP_ADD_ADDR_PORT https://git.kernel.org/netdev/net-next/c/c233ef139070 - [net-next,6/6] selftests: mptcp: add_addr and echo race test https://git.kernel.org/netdev/net-next/c/33c563ad28e3
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html