Skip to contentSkip to conversation

mptcp: fix TCP fallback on single-subflow MP_FAIL

PATCH Superseded Chenguang Zhao 6 participants 26 messages Aug 12 – Aug 14

Subsystems (2)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover mptcp: fix TCP fallback on single-subflow MP_FAIL

From: Chenguang Zhao <[email protected]>

When a valid MP_FAIL is received and infinite fallback is still allowed (single contiguous subflow), RFC8684 section 3.7 requires leaving MPTCP mode. Today the stack only clears allow_subflows and defers the real fallback to the later infinite-map transmit path. If no data is sent in between, the peer can still complete the 4th ACK as MPTCP and keep using MPTCP options.

This series adds a dedicated MPFailFallback MIB, rejects joins after fallback via mptcp_is_fully_established(), resets when the OoO queue blocks infinite mapping, and falls back immediately after the MP_FAIL response through mptcp_try_fallback().

Patch 1 adds the MPFailFallback MIB.
Patch 2 rejects joins once fallback has completed. Patch 3 resets on MP_FAIL when the OoO queue is non-empty. Patch 4 falls back to TCP after sending the MP_FAIL response.

v3:
- Address Matthieu Baerts' review

- Introduce the MIB first (Fixes: c65c2e3bae69) and reuse mptcp_try_fallback() in the functional fix

- Split mptcp_is_fully_established() and OoO handling into dedicated commits for easier backport

- Check FALLBACK_DONE via test_bit()

- Drop allow_subflows from mptcp_is_fully_established() (and the READ/WRITE_ONCE follow-up)

- On OoO failure, reset without sending MP_FAIL

- This patch series is based on the export branch, The git repository
is: git://git.kernel.org/pub/scm/linux/kernel/git/mptcp/linux.git

Chenguang Zhao (4):
mptcp: add MPFailFallback MIB
mptcp: reject joins after fallback in mptcp_is_fully_established
mptcp: reset subflow on MP_FAIL when OoO queue is non-empty
mptcp: fallback to TCP on MP_FAIL with a single subflow

 net/mptcp/mib.c      |  1 +
 net/mptcp/mib.h      |  1 +
 net/mptcp/pm.c       | 17 +++++++++++++++--
 net/mptcp/protocol.c |  4 +++-
 net/mptcp/protocol.h |  5 ++++-
 5 files changed, 24 insertions(+), 4 deletions(-)

---
v2:
https://lore.kernel.org/all/[email protected]/

v1:
https://lore.kernel.org/all/[email protected]/

Signature · 1 line
2.25.1
#1 lore raw

PATCH v3 1/4 mptcp: add MPFailFallback MIB

From: Chenguang Zhao <[email protected]>

Add a dedicated MIB to track TCP fallbacks triggered by receiving MP_FAIL. A follow-up change will use it from mptcp_pm_mp_fail_received() via mptcp_try_fallback().

Fixes c65c2e3bae69 ("mptcp: track fallbacks accurately via mibs")
---
 net/mptcp/mib.c | 1 +
 net/mptcp/mib.h | 1 +
 2 files changed, 2 insertions(+)
net/mptcp/mib.c +1 -0
index 608cb568897c..3e1bb4ff07b7 100644
@@ -90,6 +90,7 @@ static const struct snmp_mib mptcp_snmp_list[] = {
 	SNMP_MIB_ITEM("DssFallback", MPTCP_MIB_DSSFALLBACK),
 	SNMP_MIB_ITEM("DssReset", MPTCP_MIB_DSSRESET),
 	SNMP_MIB_ITEM("SimultConnectFallback", MPTCP_MIB_SIMULTCONNFALLBACK),
+	SNMP_MIB_ITEM("MPFailFallback", MPTCP_MIB_MPFAILFALLBACK),
 	SNMP_MIB_ITEM("FallbackFailed", MPTCP_MIB_FALLBACKFAILED),
 	SNMP_MIB_ITEM("WinProbe", MPTCP_MIB_WINPROBE),
 	SNMP_MIB_ITEM("BacklogDrop", MPTCP_MIB_BACKLOGDROP),
net/mptcp/mib.h +1 -0
index 1ebdb55e9534..89ebf7086cc6 100644
@@ -93,6 +93,7 @@ enum linux_mptcp_mib_field {
 	MPTCP_MIB_DSSFALLBACK,		/* Bad or missing DSS */
 	MPTCP_MIB_DSSRESET,		/* MP_RST: bad data mapping */
 	MPTCP_MIB_SIMULTCONNFALLBACK,	/* Simultaneous connect */
+	MPTCP_MIB_MPFAILFALLBACK,	/* Received MP_FAIL, fallback to TCP */
 	MPTCP_MIB_FALLBACKFAILED,	/* Can't fallback due to msk status */
 	MPTCP_MIB_WINPROBE,		/* MPTCP-level zero window probe */
 	MPTCP_MIB_BACKLOGDROP,		/* Backlog over memory limit */
Signature · 1 line
2.25.1
#2 lore raw

PATCH v3 2/4 mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to Chenguang Zhao

From: Chenguang Zhao <[email protected]>

After fallback, treat the connection as not fully established so later MP_JOIN attempts are rejected.

---
 net/mptcp/protocol.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
net/mptcp/protocol.h +4 -1
index 7e168e450fb0..bf2483a7ed92 100644
@@ -957,8 +957,11 @@ static inline void mptcp_start_tout_timer(struct sock *sk)
 
 static inline bool mptcp_is_fully_established(struct sock *sk)
 {
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
 	return inet_sk_state_load(sk) == TCP_ESTABLISHED &&
-	       READ_ONCE(mptcp_sk(sk)->fully_established);
+	       READ_ONCE(msk->fully_established) &&
+	       !test_bit(MPTCP_FALLBACK_DONE, &msk->flags);
 }
 
 static inline u64 mptcp_stamp(void)
Signature · 1 line
2.25.1
#16 lore raw

PATCH v3 3/4 mptcp: reset subflow on MP_FAIL when OoO queue is non-empty

In reply to Chenguang Zhao

From: Chenguang Zhao <[email protected]>

Infinite mapping needs contiguous data. If the MPTCP OoO queue is not empty, reset the subflow instead of falling back.

---
 net/mptcp/pm.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
net/mptcp/pm.c +10 -0
index ba7c6f80a183..8c263084db7b 100644
@@ -882,6 +882,16 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		spin_unlock_bh(&msk->fallback_lock);
 		return;
 	}
+
+	/* RFC8684 §3.7: Infinite mapping requires contiguous data */
+	if (!subflow->fail_tout &&
+	    !RB_EMPTY_ROOT(&msk->out_of_order_queue)) {
+		spin_unlock_bh(&msk->fallback_lock);
+		MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_FALLBACKFAILED);
+		mptcp_subflow_reset(sk);
+		return;
+	}
+
 	msk->allow_subflows = false;
 	spin_unlock_bh(&msk->fallback_lock);
 
Signature · 1 line
2.25.1
#20 lore raw

PATCH v3 4/4 mptcp: fallback to TCP on MP_FAIL with a single subflow

In reply to Chenguang Zhao

From: Chenguang Zhao <[email protected]>

Fall back immediately via mptcp_try_fallback() after accepting MP_FAIL on a single contiguous subflow, as required by RFC8684 §3.7.

Fixes 1e39e5a32ad7 ("mptcp: infinite mapping sending")
---
 net/mptcp/pm.c       | 9 ++++++---
 net/mptcp/protocol.c | 4 +++-
 2 files changed, 9 insertions(+), 4 deletions(-)
net/mptcp/pm.c +6 -3
index 8c263084db7b..cb85caf1df43 100644
@@ -876,7 +876,6 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 
 	pr_debug("fail_seq=%llu\n", fail_seq);
 
-	/* After accepting the fail, we can't create any other subflows */
 	spin_lock_bh(&msk->fallback_lock);
 	if (!msk->allow_infinite_fallback) {
 		spin_unlock_bh(&msk->fallback_lock);
@@ -891,8 +890,6 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		mptcp_subflow_reset(sk);
 		return;
 	}
-
-	msk->allow_subflows = false;
 	spin_unlock_bh(&msk->fallback_lock);
 
 	if (!subflow->fail_tout) {
@@ -901,6 +898,12 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		subflow->send_mp_fail = 1;
 		subflow->send_infinite_map = 1;
 		tcp_send_ack(sk);
+
+		/* RFC8684 §3.7: fallback with a single subflow */
+		if (!mptcp_try_fallback(sk, MPTCP_MIB_MPFAILFALLBACK)) {
+			MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_FALLBACKFAILED);
+			mptcp_subflow_reset(sk);
+		}
 	} else {
 		pr_debug("MP_FAIL response received\n");
 		WRITE_ONCE(subflow->fail_tout, 0);
net/mptcp/protocol.c +3 -1
index f879b1061f2d..519e8d9c3164 100644
@@ -1418,7 +1418,9 @@ static void mptcp_update_infinite_map(struct mptcp_sock *msk,
 	mpext->infinite_map = 1;
 	mpext->data_len = 0;
 
-	if (!mptcp_try_fallback(ssk, MPTCP_MIB_INFINITEMAPTX)) {
+	if (__mptcp_check_fallback(msk)) {
+		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_INFINITEMAPTX);
+	} else if (!mptcp_try_fallback(ssk, MPTCP_MIB_INFINITEMAPTX)) {
 		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_FALLBACKFAILED);
 		mptcp_subflow_reset(ssk);
 		return;
Signature · 1 line
2.25.1
#17 lore raw

Re: [PATCH export v3 3/4] mptcp: reset subflow on MP_FAIL when OoO queue is non-empty

In reply to Chenguang Zhao
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] TOCTOU race due to lockless access of `msk->out_of_order_queue`. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] TOCTOU race due to lockless access of msk->out_of_order_queue. --

--- Patch [3]: [PATCH export v3 3/4] mptcp: reset subflow on MP_FAIL when OoO queue is non-empty --- commit fe6cef2f543d25c806aed2ee11c8a4d7a936c259
Author: Chenguang Zhao <[email protected]>

mptcp: reset subflow on MP_FAIL when OoO queue is non-empty

This commit updates the handling of an MP_FAIL response to reset the subflow instead of falling back when the out-of-order queue is not empty. This is done to ensure the contiguous data requirements for infinite mapping are met.

net/mptcp/pm.c void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq) · line 888

Quoted 12 lines
net/mptcp/pm.c +4 -0
index ba7c6f80a183c..8c263084db7be 100644
@@ -882,6 +882,16 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		spin_unlock_bh(&msk->fallback_lock);
 		return;
 	}
+
+	/* RFC8684 §3.7: Infinite mapping requires contiguous data */
+	if (!subflow->fail_tout &&
+	    !RB_EMPTY_ROOT(&msk->out_of_order_queue)) {

[Severity: High]
Can this lockless check of the out-of-order queue lead to a race condition?

Looking at mptcp_pm_mp_fail_received(), the out_of_order_queue is checked for emptiness while only holding msk->fallback_lock, without the required mptcp_data_lock.

If another active subflow concurrently receives out-of-order data, acquires the msk data lock, and inserts data into the queue, this check might incorrectly conclude the queue is empty.

If the queue is populated right after this lockless check but before mptcp_try_fallback() is called, could this cause an infinite map TCP ACK to be incorrectly transmitted before the fallback is ultimately aborted?

Furthermore, if the queue is populated immediately after fallback occurs, could this stranded MPTCP out-of-order data be improperly appended to the plain TCP receive queue as the stream advances, resulting in data duplication and stream corruption?

Quoted 8 lines
net/mptcp/pm.c +6 -0
+		spin_unlock_bh(&msk->fallback_lock);
+		MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_FALLBACKFAILED);
+		mptcp_subflow_reset(sk);
+		return;
+	}
+
 	msk->allow_subflows = false;
 	spin_unlock_bh(&msk->fallback_lock);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
#25 lore raw
In reply to Chenguang Zhao

Hi Chenguang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/31568808826

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/185641831778
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1144463

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

maintainer
#3 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to Chenguang Zhao
Quoted 15 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

After fallback, treat the connection as not fully established so later MP_JOIN attempts are rejected.

---
 net/mptcp/protocol.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
net/mptcp/protocol.h +0 -0
index 7e168e450fb0..bf2483a7ed92 100644

net/mptcp/protocol.h static inline void mptcp_start_tout_timer(struct sock *sk) · line 964

Quoted 9 lines
net/mptcp/protocol.h +4 -1
@@ -957,8 +957,11 @@ static inline void mptcp_start_tout_timer(struct sock *sk)
 
 static inline bool mptcp_is_fully_established(struct sock *sk)
 {
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
 	return inet_sk_state_load(sk) == TCP_ESTABLISHED &&
-	       READ_ONCE(mptcp_sk(sk)->fully_established);
+	       READ_ONCE(msk->fully_established) &&
+	       !test_bit(MPTCP_FALLBACK_DONE, &msk->flags);

Does the above improve actually anything? The test is inherently racy, as lack the fallback_lock, and AFAICS all critical paths have already explicit checks under such lock, see i.e. mptcp_finish_join().

I would prefer avoiding additional conditionals, if not well reasoned.

/P

maintainer
#18 lore raw

Re: [PATCH export v3 3/4] mptcp: reset subflow on MP_FAIL when OoO queue is non-empty

In reply to Chenguang Zhao
Quoted 4 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

Infinite mapping needs contiguous data. If the MPTCP OoO queue is not empty, reset the subflow instead of falling back.

Why? the RFC says:

"""if all unacknowledged data in flight is contiguous"""

That statement can be true even when the RTX queue is not empty.

It will become false if/when some mptcp-level retransmission will happen.

__mptcp_push_retrans() atomically sets allow_infinite_fallback to false when such retransmission happen. The code just before the newly added one checks such condition under the relevant lock.

I don't think this change is needed.

/P

maintainer
#21 lore raw

Re: [PATCH export v3 4/4] mptcp: fallback to TCP on MP_FAIL with a single subflow

In reply to Chenguang Zhao
Quoted 25 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

Fall back immediately via mptcp_try_fallback() after accepting MP_FAIL on a single contiguous subflow, as required by RFC8684 §3.7.

Fixes 1e39e5a32ad7 ("mptcp: infinite mapping sending")
---
 net/mptcp/pm.c       | 9 ++++++---
 net/mptcp/protocol.c | 4 +++-
 2 files changed, 9 insertions(+), 4 deletions(-)
net/mptcp/pm.c +0 -1
index 8c263084db7b..cb85caf1df43 100644
@@ -876,7 +876,6 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 
 	pr_debug("fail_seq=%llu\n", fail_seq);
 
-	/* After accepting the fail, we can't create any other subflows */
 	spin_lock_bh(&msk->fallback_lock);
 	if (!msk->allow_infinite_fallback) {
 		spin_unlock_bh(&msk->fallback_lock);

net/mptcp/pm.c void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq) · line 893

Quoted 6 lines
net/mptcp/pm.c +0 -2
@@ -891,8 +890,6 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		mptcp_subflow_reset(sk);
 		return;
 	}
-
-	msk->allow_subflows = false;
 	spin_unlock_bh(&msk->fallback_lock);

At this point another subflow can complete the join, and set allow_infinite_fallback = false ...

net/mptcp/pm.c void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq) · line 900

Quoted 6 lines
net/mptcp/pm.c +0 -0
 	if (!subflow->fail_tout) {
@@ -901,6 +898,12 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		subflow->send_mp_fail = 1;
 		subflow->send_infinite_map = 1;
 		tcp_send_ack(sk);

... so the this mp_fail processing will be bogus [1].

Quoted 15 lines
net/mptcp/pm.c +6 -0
+
+		/* RFC8684 §3.7: fallback with a single subflow */
+		if (!mptcp_try_fallback(sk, MPTCP_MIB_MPFAILFALLBACK)) {
+			MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_FALLBACKFAILED);
+			mptcp_subflow_reset(sk);
+		}
 
 	} else {
 		pr_debug("MP_FAIL response received\n");
net/mptcp/pm.c +0 -0
 		WRITE_ONCE(subflow->fail_tout, 0);
net/mptcp/protocol.c +0 -0
index f879b1061f2d..519e8d9c3164 100644

net/mptcp/protocol.c static void mptcp_update_infinite_map(struct mptcp_sock *msk, · line 1426

Quoted 10 lines
net/mptcp/protocol.c +3 -1
@@ -1418,7 +1418,9 @@ static void mptcp_update_infinite_map(struct mptcp_sock *msk,
 	mpext->infinite_map = 1;
 	mpext->data_len = 0;
 
-	if (!mptcp_try_fallback(ssk, MPTCP_MIB_INFINITEMAPTX)) {
+	if (__mptcp_check_fallback(msk)) {
+		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_INFINITEMAPTX);
+	} else if (!mptcp_try_fallback(ssk, MPTCP_MIB_INFINITEMAPTX)) {
 		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_FALLBACKFAILED);
 		mptcp_subflow_reset(ssk);
 		return;

I don't understand this change. Can we ever enter the

if (!mptcp_try_fallback(ssk, MPTCP_MIB_INFINITEMAPTX)) {

branch? The msk already tried to fallback in mptcp_pm_mp_fail_received(). If the fallback was successful, the code will enter the if (__mptcp_check_fallback(msk)) { branch and not this one.

Otherwise the fallback will fail again (AFAICS nothing resets allow_infinite_fallback once in become false).

/P

maintainer
#23 lore raw

Re: [PATCH export v3 4/4] mptcp: fallback to TCP on MP_FAIL with a single subflow

In reply to Chenguang Zhao

net/mptcp/pm.c void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq) · line 900

Quoted 4 lines · Chenguang Zhao
net/mptcp/pm.c +0 -0
@@ -901,6 +898,12 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		subflow->send_mp_fail = 1;
 		subflow->send_infinite_map = 1;
 		tcp_send_ack(sk);

Slightly unrelated... AFAICS the current code sends the mp_fail reply immediatally, and will send the infinite mapping with the later data send, if any.

I *think* it will be better to send MP_FAIL and infinite mapping together. It should remove a little bit of complexity on the xmit path and will be IMHO closer to the RFC specs.

/P

maintainer
#4 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to Paolo Abeni

Hi Paolo, Chenguang,

Quoted 5 lines · Paolo Abeni
Quoted 4 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

After fallback, treat the connection as not fully established so later MP_JOIN attempts are rejected.

@Chenguang: please first reply to the questions and comments from the previous versions, then only send a new version when the discussions are over.

Quoted 11 lines
Quoted 11 lines
---
 net/mptcp/protocol.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
net/mptcp/protocol.h +0 -0
index 7e168e450fb0..bf2483a7ed92 100644
@@ -957,8 +957,11 @@ static inline void mptcp_start_tout_timer(struct sock *sk)
 
Quoted 12 lines

net/mptcp/protocol.h static inline void mptcp_start_tout_timer(struct sock *sk) · line 964

Quoted 8 lines
net/mptcp/protocol.h +4 -1
 static inline bool mptcp_is_fully_established(struct sock *sk)
 {
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
 	return inet_sk_state_load(sk) == TCP_ESTABLISHED &&
-	       READ_ONCE(mptcp_sk(sk)->fully_established);
+	       READ_ONCE(msk->fully_established) &&
+	       !test_bit(MPTCP_FALLBACK_DONE, &msk->flags);

Does the above improve actually anything? The test is inherently racy, as lack the fallback_lock, and AFAICS all critical paths have already explicit checks under such lock, see i.e. mptcp_finish_join().

Indeed, the fallback lock is missing.

Note: Chenguang is looking at a bug where, when an MP_FAIL is received before being in fully_established, the kernel accept going to fully established, and creating new subflows, then reject them once done.

Same when receiving a 4th packet without MPTCP options.

I think the MP_FAIL should do a fallback if possible, but it is probably also required to check the fallback status or something similar when receiving an MPJ.

Quoted 1 line

I would prefer avoiding additional conditionals, if not well reasoned.

Agreed, the commit messages in this series are not explaining (enough) the reason and the context.

BTW, Chenguang, please use 'mptcp-net' for fixes (and mptcp-next for features). Also fixes should have a Fixes tag.

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
#14 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to Paolo Abeni
Quoted 29 lines · Paolo Abeni
Quoted 15 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

After fallback, treat the connection as not fully established so later MP_JOIN attempts are rejected.

---
 net/mptcp/protocol.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
net/mptcp/protocol.h +0 -0
index 7e168e450fb0..bf2483a7ed92 100644

net/mptcp/protocol.h static inline void mptcp_start_tout_timer(struct sock *sk) · line 964

Quoted 9 lines
net/mptcp/protocol.h +4 -1
@@ -957,8 +957,11 @@ static inline void mptcp_start_tout_timer(struct sock *sk)
 
 static inline bool mptcp_is_fully_established(struct sock *sk)
 {
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
 	return inet_sk_state_load(sk) == TCP_ESTABLISHED &&
-	       READ_ONCE(mptcp_sk(sk)->fully_established);
+	       READ_ONCE(msk->fully_established) &&
+	       !test_bit(MPTCP_FALLBACK_DONE, &msk->flags);

Does the above improve actually anything? The test is inherently racy, as lack the fallback_lock, and AFAICS all critical paths have already explicit checks under such lock, see i.e. mptcp_finish_join().

Quoted 4 lines

I would prefer avoiding additional conditionals, if not well reasoned.

/P

Hi Paolo

You are right: the extra FALLBACK_DONE check in mptcp_is_fully_established() does not actually improve anything. It is read without fallback_lock, so it is racy. The paths that must not complete a join after MP_FAIL already reject it under that lock via allow_subflows, e.g. mptcp_finish_join() / __mptcp_finish_join(). I will drop this patch in the next revision and leave mptcp_is_fully_established() unchanged.

Thanks,
Chenguang

#19 lore raw

Re: [PATCH export v3 3/4] mptcp: reset subflow on MP_FAIL when OoO queue is non-empty

In reply to Paolo Abeni
Quoted 13 lines · Paolo Abeni
Quoted 4 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

Infinite mapping needs contiguous data. If the MPTCP OoO queue is not empty, reset the subflow instead of falling back.

Why? the RFC says:

"""if all unacknowledged data in flight is contiguous"""

That statement can be true even when the RTX queue is not empty.

It will become false if/when some mptcp-level retransmission will happen.

Quoted 8 lines

__mptcp_push_retrans() atomically sets allow_infinite_fallback to false when such retransmission happen. The code just before the newly added one checks such condition under the relevant lock.

I don't think this change is needed.

/P

Hi Paolo

I misread the RFC here. "If all unacknowledged data in flight is contiguous" is about send-side in-flight data, not about the MPTCP receive OoO queue being empty. Contiguous in-flight data can still hold when the RTX queue is not empty; it only becomes false after an MPTCP-level retransmission.

That is already tracked by allow_infinite_fallback, which __mptcp_push_retrans() clears under fallback_lock. The existing check of that flag at the start of mptcp_pm_mp_fail_received() covers the RFC condition.

I will drop this patch in the next revision.

Thanks,
Chenguang

#22 lore raw

Re: [PATCH export v3 4/4] mptcp: fallback to TCP on MP_FAIL with a single subflow

In reply to Paolo Abeni
Quoted 22 lines · Paolo Abeni
Quoted 20 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

Fall back immediately via mptcp_try_fallback() after accepting MP_FAIL on a single contiguous subflow, as required by RFC8684 §3.7.

Fixes 1e39e5a32ad7 ("mptcp: infinite mapping sending")
---
 net/mptcp/pm.c       | 9 ++++++---
 net/mptcp/protocol.c | 4 +++-
 2 files changed, 9 insertions(+), 4 deletions(-)
net/mptcp/pm.c +0 -0
index 8c263084db7b..cb85caf1df43 100644
@@ -876,7 +876,6 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 
 	pr_debug("fail_seq=%llu\n", fail_seq);
 
Quoted 13 lines

net/mptcp/pm.c void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq) · line 893

Quoted 11 lines
net/mptcp/pm.c +0 -3
-	/* After accepting the fail, we can't create any other subflows */
 	spin_lock_bh(&msk->fallback_lock);
 	if (!msk->allow_infinite_fallback) {
 		spin_unlock_bh(&msk->fallback_lock);
@@ -891,8 +890,6 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		mptcp_subflow_reset(sk);
 		return;
 	}
-
-	msk->allow_subflows = false;
 	spin_unlock_bh(&msk->fallback_lock);

At this point another subflow can complete the join, and set allow_infinite_fallback = false ...

Agreed. I should not have dropped the allow_subflows = false update under fallback_lock. Join completion already checks that flag under the same lock, so clearing it before unlocking is what prevents a

concurrent join from sneaking in.

I plan to keep that assignment, then send the MP_FAIL ACK and call mptcp_try_fallback(). FALLBACK_DONE still has to come after tcp_send_ack(), otherwise mptcp_established_options() would drop the MPTCP options on that ACK.

Quoted 36 lines

net/mptcp/pm.c void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq) · line 900

Quoted 6 lines
net/mptcp/pm.c +0 -0
 	if (!subflow->fail_tout) {
@@ -901,6 +898,12 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		subflow->send_mp_fail = 1;
 		subflow->send_infinite_map = 1;
 		tcp_send_ack(sk);

... so the this mp_fail processing will be bogus [1].

Quoted 14 lines
net/mptcp/pm.c +6 -0
+
+		/* RFC8684 §3.7: fallback with a single subflow */
+		if (!mptcp_try_fallback(sk, MPTCP_MIB_MPFAILFALLBACK)) {
+			MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_FALLBACKFAILED);
+			mptcp_subflow_reset(sk);
+		}
 	} else {
 		pr_debug("MP_FAIL response received\n");
 		WRITE_ONCE(subflow->fail_tout, 0);
net/mptcp/protocol.c +0 -0
index f879b1061f2d..519e8d9c3164 100644

net/mptcp/protocol.c static void mptcp_update_infinite_map(struct mptcp_sock *msk, · line 1426

Quoted 10 lines
net/mptcp/protocol.c +3 -1
@@ -1418,7 +1418,9 @@ static void mptcp_update_infinite_map(struct mptcp_sock *msk,
 	mpext->infinite_map = 1;
 	mpext->data_len = 0;
 
-	if (!mptcp_try_fallback(ssk, MPTCP_MIB_INFINITEMAPTX)) {
+	if (__mptcp_check_fallback(msk)) {
+		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_INFINITEMAPTX);
+	} else if (!mptcp_try_fallback(ssk, MPTCP_MIB_INFINITEMAPTX)) {
 		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_FALLBACKFAILED);
 		mptcp_subflow_reset(ssk);
 		return;

I don't understand this change. Can we ever enter the

if (!mptcp_try_fallback(ssk, MPTCP_MIB_INFINITEMAPTX)) {

Quoted 8 lines

branch? The msk already tried to fallback in mptcp_pm_mp_fail_received(). If the fallback was successful, the code will enter the if (__mptcp_check_fallback(msk)) { branch and not this one.

Otherwise the fallback will fail again (AFAICS nothing resets allow_infinite_fallback once in become false).

/P

No. After moving the fallback to mptcp_pm_mp_fail_received(), that failure path in mptcp_update_infinite_map() is not reachable in the normal case: either FALLBACK_DONE is already set, or the earlier try_fallback() already failed and reset the subflow. allow_infinite_fallback is not set back to true on that connection.

try_fallback(INFINITEMAPTX) also cannot be used just to account for the map: __mptcp_try_fallback() returns true immediately when fallback is already done and does not increment the MIB.

I would drop that second try_fallback(). Fallback would be counted as MPFailFallback when MP_FAIL is accepted; InfiniteMapTx would be incremented when the infinite mapping is actually transmitted later.

#24 lore raw

Re: [PATCH export v3 4/4] mptcp: fallback to TCP on MP_FAIL with a single subflow

In reply to Paolo Abeni
Quoted 9 lines · Paolo Abeni

net/mptcp/pm.c void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq) · line 900

Quoted 4 lines · Chenguang Zhao
net/mptcp/pm.c +0 -0
@@ -901,6 +898,12 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		subflow->send_mp_fail = 1;
 		subflow->send_infinite_map = 1;
 		tcp_send_ack(sk);

Slightly unrelated... AFAICS the current code sends the mp_fail reply immediatally, and will send the infinite mapping with the later data send, if any.

Quoted 6 lines

I *think* it will be better to send MP_FAIL and infinite mapping together. It should remove a little bit of complexity on the xmit path and will be IMHO closer to the RFC specs.

/P

To keep both MPTCP options on the same packet, that packet would have to be sent before FALLBACK_DONE: once fallback is done, mptcp_established_options() drops MPTCP options unless the skb already carries an infinite mapping.

tcp_send_ack() builds a plain ACK with no mpext->infinite_map, so it can include MP_FAIL but not the infinite mapping. The infinite mapping is filled in later on a data skb in mptcp_sendmsg_frag(). With the current xmit path those two options therefore cannot go out on the same packet. Sending them together would need a dedicated transmit helper that builds a skb carrying both, rather than reusing tcp_send_ack().

#5 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to Matthieu Baerts
Quoted 10 lines · Matthieu Baerts

Note: Chenguang is looking at a bug where, when an MP_FAIL is received before being in fully_established, the kernel accept going to fully established, and creating new subflows, then reject them once done.

Same when receiving a 4th packet without MPTCP options.

I think the MP_FAIL should do a fallback if possible, but it is probably also required to check the fallback status or something similar when receiving an MPJ.

Hi, Matt

Sorry, I'm not very familiar with the details of this patch.

No offense intended, I just have a question I'd like to ask based on my understanding and the RFC documentation: Should MP_FAIL trigger a fallback?

For the multiple‑subflow case, if one subflow encounters an MP_FAIL, it should be closed with a RST, and the data will be retransmitted over the other healthy subflows. This should not cause the entire MPTCP connection to fallback to regular TCP, right?
RFC8684's contents:
'''
The receiver of this option MUST discard all data following the data sequence number specified. Failed data MUST NOT be DATA_ACKed and so will be retransmitted on other subflows. '''

For the single‑subflow case, according to the RFC, MP_FAIL should subsequently lead to a fallback under the infinite mapping (INFINITEMAP) situation, shouldn't it? RFC 8684's contents:
'''
A special case is when there is a single subflow and it fails with a checksum error. If it is known that all unacknowledged data in flight is contiguous (which will usually be the case with a single subflow), an infinite mapping can be applied to the subflow without the need to close it first, essentially turning off all further MPTCP signaling. '''

Thanks
Gang

Quoted 15 lines
Quoted 4 lines

I would prefer avoiding additional conditionals, if not well reasoned.

Agreed, the commit messages in this series are not explaining (enough) the reason and the context.

BTW, Chenguang, please use 'mptcp-net' for fixes (and mptcp-next for features). Also fixes should have a Fixes tag.

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
maintainer
#6 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

Hi Gang,

Quoted 21 lines · On 13/08/2026 11:32, [email protected] wrote:
Quoted 10 lines · Matthieu Baerts

Note: Chenguang is looking at a bug where, when an MP_FAIL is received before being in fully_established, the kernel accept going to fully established, and creating new subflows, then reject them once done.

Same when receiving a 4th packet without MPTCP options.

I think the MP_FAIL should do a fallback if possible, but it is probably also required to check the fallback status or something similar when receiving an MPJ.

Hi, Matt

Sorry, I'm not very familiar with the details of this patch.

No offense intended, I just have a question I'd like to ask based on my understanding and the RFC documentation: Should MP_FAIL trigger a fallback?

Quoted 9 lines

For the multiple‑subflow case, if one subflow encounters an MP_FAIL, it should be closed with a RST, and the data will be retransmitted over the other healthy subflows. This should not cause the entire MPTCP connection to fallback to regular TCP, right?
RFC8684's contents:
'''
The receiver of this option MUST discard all data following the data sequence number specified. Failed data MUST NOT be DATA_ACKed and so will be retransmitted on other subflows. '''

Correct, in this case it shouldn't fallback.

Quoted 9 lines

For the single‑subflow case, according to the RFC, MP_FAIL should subsequently lead to a fallback under the infinite mapping (INFINITEMAP) situation, shouldn't it? RFC 8684's contents:
'''
A special case is when there is a single subflow and it fails with a checksum error. If it is known that all unacknowledged data in flight is contiguous (which will usually be the case with a single subflow), an infinite mapping can be applied to the subflow without the need to close it first, essentially turning off all further MPTCP signaling. '''

Correct.

Please note that when I said "the MP_FAIL should [cause] a fallback if possible", I'm referring to what is done in mptcp_try_fallback() → a fallback is only possible in the single subflow situation, without reordering at the MPTCP level.

In addition, in the case we are trying to fix here, the MP_FAIL is received before switching to fully established: the RFC doesn't explain this particular case, but it sounds like there is no need to send the infinite mapping, and a fallback can be done directly *I think*. The thing is this situation shouldn't happen: we are not supposed to receive an MP_FAIL at that stage, no data has been exchanged at the MPTCP level, so let's do the minimal -- i.e. just a fallback -- to avoid issues and reduce the complexity, no?

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
maintainer
#15 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to Chenguang Zhao

Hi Chenguang,

Quoted 37 lines · Chenguang Zhao
Quoted 29 lines · Paolo Abeni
Quoted 15 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

After fallback, treat the connection as not fully established so later MP_JOIN attempts are rejected.

---
 net/mptcp/protocol.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
net/mptcp/protocol.h +0 -0
index 7e168e450fb0..bf2483a7ed92 100644

net/mptcp/protocol.h static inline void mptcp_start_tout_timer(struct sock *sk) · line 964

Quoted 9 lines
net/mptcp/protocol.h +4 -1
@@ -957,8 +957,11 @@ static inline void mptcp_start_tout_timer(struct sock *sk)
 
 static inline bool mptcp_is_fully_established(struct sock *sk)
 {
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
 	return inet_sk_state_load(sk) == TCP_ESTABLISHED &&
-	       READ_ONCE(mptcp_sk(sk)->fully_established);
+	       READ_ONCE(msk->fully_established) &&
+	       !test_bit(MPTCP_FALLBACK_DONE, &msk->flags);

Does the above improve actually anything? The test is inherently racy, as lack the fallback_lock, and AFAICS all critical paths have already explicit checks under such lock, see i.e. mptcp_finish_join().

Quoted 4 lines

I would prefer avoiding additional conditionals, if not well reasoned.

/P

Hi Paolo

Quoted 7 lines

You are right: the extra FALLBACK_DONE check in mptcp_is_fully_established() does not actually improve anything. It is read without fallback_lock, so it is racy. The paths that must not complete a join after MP_FAIL already reject it under that lock via allow_subflows, e.g. mptcp_finish_join() / __mptcp_finish_join(). I will drop this patch in the next revision and leave mptcp_is_fully_established() unchanged.

How do you plan to deal with the case where a fallback has been done while being in fully established mode?

Could we add WRITE_ONCE(msk->fully_established, false) in __mptcp_try_fallback()? I guess it is acceptable because __mptcp_finish_join will check allow_subflows under lock, no?

Or maybe we accept this behaviour?

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
#7 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to Matthieu Baerts
Quoted 10 lines · Matthieu Baerts

In addition, in the case we are trying to fix here, the MP_FAIL is received before switching to fully established: the RFC doesn't explain this particular case, but it sounds like there is no need to send the infinite mapping, and a fallback can be done directly *I think*. The thing is this situation shouldn't happen: we are not supposed to receive an MP_FAIL at that stage, no data has been exchanged at the MPTCP level, so let's do the minimal -- i.e. just a fallback -- to avoid issues and reduce the complexity, no?

Hi Matt,

Thanks for your explanation. I have a quick thought:

Can we check the 'subflow->send_mp_fail' in 'subflow_finish_connect'? like:

'''
...
543 if (subflow->request_mptcp) {
544 if (!(mp_opt.suboptions & OPTION_MPTCP_MPC_SYNACK)) {
545 if (!mptcp_try_fallback(sk,
546 MPTCP_MIB_MPCAPABLEACTIVEFALLBACK)) {
547 MPTCP_INC_STATS(sock_net(sk),
548 MPTCP_MIB_FALLBACKFAILED);
549 goto do_reset;
550 }
551
552 goto fallback;
553 }

             if (subflow->send_mp_fail)
                             goto fallback;

554
...
'''

Or, check msk->fully_established in pm_fail_received like this:
'''
@@ -901,6 +898,12 @@ void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
subflow->send_mp_fail = 1;
subflow->send_infinite_map = 1;
tcp_send_ack(sk);
+
+ if (!msk->fully_established && !mptcp_try_fallback(sk, MPTCP_MIB_MPFAILFALLBACK)) {
+ MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_FALLBACKFAILED);
+ mptcp_subflow_reset(sk);
+ }
'''

Because the MP_FAIL specified in the RFC does not seem to actively trigger a fallback on its own; instead, it is triggered via the infinite mapping. If we modify the behavior of MP_FAIL in the regular path, I think it might be over-design and could introduce additional risks. What do you think?

Thanks
Gang

Quoted 5 lines

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
maintainer
#9 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to Matthieu Baerts
Quoted 37 lines · Matthieu Baerts

Hi Paolo, Chenguang,

Quoted 5 lines · Paolo Abeni
Quoted 4 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

After fallback, treat the connection as not fully established so later MP_JOIN attempts are rejected.

@Chenguang: please first reply to the questions and comments from the previous versions, then only send a new version when the discussions are over.

Quoted 11 lines
Quoted 11 lines
---
 net/mptcp/protocol.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
net/mptcp/protocol.h +0 -0
index 7e168e450fb0..bf2483a7ed92 100644
@@ -957,8 +957,11 @@ static inline void mptcp_start_tout_timer(struct sock *sk)
 
Quoted 12 lines

net/mptcp/protocol.h static inline void mptcp_start_tout_timer(struct sock *sk) · line 964

Quoted 8 lines
net/mptcp/protocol.h +4 -1
 static inline bool mptcp_is_fully_established(struct sock *sk)
 {
+	struct mptcp_sock *msk = mptcp_sk(sk);
+
 	return inet_sk_state_load(sk) == TCP_ESTABLISHED &&
-	       READ_ONCE(mptcp_sk(sk)->fully_established);
+	       READ_ONCE(msk->fully_established) &&
+	       !test_bit(MPTCP_FALLBACK_DONE, &msk->flags);

Does the above improve actually anything? The test is inherently racy, as lack the fallback_lock, and AFAICS all critical paths have already explicit checks under such lock, see i.e. mptcp_finish_join().

Quoted 7 lines

Indeed, the fallback lock is missing.

Note: Chenguang is looking at a bug where, when an MP_FAIL is received before being in fully_established, the kernel accept going to fully established, and creating new subflows, then reject them once done.

Same when receiving a 4th packet without MPTCP options.

It's not clear to me what actually goes wrong and how. Do you have handy a event sequence or stack trace demonstrating the critical scenario?

Both the above look like bugs in check_fully_established(), or incorrect/later check of fallback status/fallback_lock.

i.e. AFAICS active subflows are created and added to the conn_list without checking the fallback status/fallback_lock, but the later join correctly rejects the subflow finalization, so there should any real problems beyond some noise.

/P

maintainer
#10 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

Hi Paolo,

Thank you for your reply!

14 Aug 2026 10:43:41 Paolo Abeni <[email protected]>:

Quoted 42 lines · Matthieu Baerts
Quoted 39 lines

Hi Paolo, Chenguang,

Quoted 5 lines · Paolo Abeni
Quoted 4 lines · Chenguang Zhao

From: Chenguang Zhao <[email protected]>

After fallback, treat the connection as not fully established so later MP_JOIN attempts are rejected.

@Chenguang: please first reply to the questions and comments from the previous versions, then only send a new version when the discussions are over.

Quoted 11 lines
Quoted 11 lines

---
net/mptcp/protocol.h | 5 ++++-

1 file changed, 4 insertions(+), 1 deletion(-)
net/mptcp/protocol.h +0 -0
index 7e168e450fb0..bf2483a7ed92 100644
@@ -957,8 +957,11 @@ static inline void mptcp_start_tout_timer(struct sock *sk)
 
Quoted 12 lines
Quoted 8 lines
net/mptcp/protocol.h +2 -0
 static inline bool mptcp_is_fully_established(struct sock *sk)
 {
+   struct mptcp_sock *msk = mptcp_sk(sk);
+

    return inet_sk_state_load(sk) == TCP_ESTABLISHED && -          READ_ONCE(mptcp_sk(sk)->fully_established); +          READ_ONCE(msk->fully_established) && +          !test_bit(MPTCP_FALLBACK_DONE, &msk->flags);

Does the above improve actually anything? The test is inherently racy, as lack the fallback_lock, and AFAICS all critical paths have already explicit checks under such lock, see i.e.  mptcp_finish_join().

Indeed, the fallback lock is missing.

Quoted 8 lines
Quoted 5 lines

Note: Chenguang is looking at a bug where, when an MP_FAIL is received before being in fully_established, the kernel accept going to fully established, and creating new subflows, then reject them once done.

Same when receiving a 4th packet without MPTCP options.

It's not clear to me what actually goes wrong and how. Do you have handy a event sequence or stack trace demonstrating the critical scenario?

Indeed, better with packetdrill scripts:

https://github.com/chrisocean716-star/packetdrill/blob/bf9be7a7bee3687500444b284a9db2a0009a5252/gtests/net/mptcp/dss/dss_drop_after_data_fallback_server.pkt

=> A fallback is done, a subflow can be initiated, then rejected.

https://github.com/chrisocean716-star/packetdrill/blob/fed57fab92c407eb433acdb8c8ce72bd8e3e2f37/gtests/net/mptcp/mp_join/mp_join_server_after_fallback.pkt

=> The MP_FAIL is partially ignored if received before being in fully established state.

Quoted 7 lines

Both the above look like bugs in check_fully_established(), or incorrect/later check of fallback status/fallback_lock.

i.e. AFAICS active subflows are created and added to the conn_list without checking the fallback status/fallback_lock, but the later join correctly rejects the subflow finalization, so there should any real problems beyond some noise.

For the fallback, it feels wrong but we could say that it's just some noise.

For the MP_FAIL, we should probably not ignore it in this case.

I think the MP_FAIL case should fallback. For both, the fallback should either disable the fully established state, or the fully established helpers should look for fallback.

Cheers,
Matt

maintainer
#11 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

Quoted 8 lines · Matthieu Baerts

14 Aug 2026 10:43:41 Paolo Abeni <[email protected]>:

Quoted 2 lines

It's not clear to me what actually goes wrong and how. Do you have handy a event sequence or stack trace demonstrating the critical scenario?

Indeed, better with packetdrill scripts:

https://github.com/chrisocean716-star/packetdrill/blob/bf9be7a7bee3687500444b284a9db2a0009a5252/gtests/net/mptcp/dss/dss_drop_after_data_fallback_server.pkt

Quoted 7 lines

=> A fallback is done, a subflow can be initiated, then rejected.

https://github.com/chrisocean716-star/packetdrill/blob/fed57fab92c407eb433acdb8c8ce72bd8e3e2f37/gtests/net/mptcp/mp_join/mp_join_server_after_fallback.pkt

=> The MP_FAIL is partially ignored if received before being in fully established state.

Thanks!

I'm pretty sure there is a miss-intepretation of the 2nd case. The problem is not with msk->fully_established. AFAICS msk->fully_established is true when the MP_FAIL is receive.

The passive msk socket sets it at MPC ACK reception.

The problem is that, at MP_FAIL reception, the infinite mapping is not sent immediately, but it's scheduled with the next xmit, and until such point the msk does not perform the fallback.

I think a proper solution would be bounding MP_FAIL (reply) and infinite mapping in the same ack.

As noted by Gang, it may require a new helper (build on top of the tcp_send_ack).

/P

maintainer
#8 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to [email protected]

Hi Gang,

Quoted 31 lines · On 14/08/2026 05:02, [email protected] wrote:
Quoted 10 lines · Matthieu Baerts

In addition, in the case we are trying to fix here, the MP_FAIL is received before switching to fully established: the RFC doesn't explain this particular case, but it sounds like there is no need to send the infinite mapping, and a fallback can be done directly *I think*. The thing is this situation shouldn't happen: we are not supposed to receive an MP_FAIL at that stage, no data has been exchanged at the MPTCP level, so let's do the minimal -- i.e. just a fallback -- to avoid issues and reduce the complexity, no?

Hi Matt,

Thanks for your explanation. I have a quick thought:

Can we check the 'subflow->send_mp_fail' in 'subflow_finish_connect'? like:

'''
...
543 if (subflow->request_mptcp) {
544 if (!(mp_opt.suboptions & OPTION_MPTCP_MPC_SYNACK)) {
545 if (!mptcp_try_fallback(sk,
546 MPTCP_MIB_MPCAPABLEACTIVEFALLBACK)) {
547 MPTCP_INC_STATS(sock_net(sk),
548 MPTCP_MIB_FALLBACKFAILED);
549 goto do_reset;
550 }
551
552 goto fallback;
553 }

Quoted 5 lines
             if (subflow->send_mp_fail)
                             goto fallback;

554
...
'''

I don't know: it doesn't feel right to check this field there. Sending the MP_FAIL reply and the infinite mapping in the same ACK, as you suggested, seems better.

Quoted 11 lines

Or, check msk->fully_established in pm_fail_received like this:
'''

(quoted diff) +5 -0
@@ -901,6 +898,12 @@  void mptcp_pm_mp_fail_received(struct sock *sk, u64 fail_seq)
 		subflow->send_mp_fail = 1;
 		subflow->send_infinite_map = 1;
 		tcp_send_ack(sk);
+
+		if (!msk->fully_established && !mptcp_try_fallback(sk, MPTCP_MIB_MPFAILFALLBACK)) {
+			MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_FALLBACKFAILED);
+			mptcp_subflow_reset(sk);
+		}
Quoted 1 line

'''

Paolo noticed (in another email from this thread) this will not cover the case we are trying to fix as we were already in fully_established.

But maybe we could try to fallback if that's possible, and if checksum was not being used? I didn't check if it would be a good solution.

Quoted 4 lines

Because the MP_FAIL specified in the RFC does not seem to actively trigger a fallback on its own; instead, it is triggered via the infinite mapping. If we modify the behavior of MP_FAIL in the regular path, I think it might be over-design and could introduce additional risks. What do you think?

Yes, but here, I'm only proposing doing a fallback for the cases not covered by the RFC, i.e. single subflow, no checksum.

I think the simplest solution should be picked for this use-case that shouldn't happen. Bounding MP_FAIL reply and infinite mapping in the same ack seems interesting, and not just for this case, but it might be "hacky" as you don't have an mpext in this case, no? Maybe a fallback is "cleaner"?

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
maintainer
#12 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

In reply to Paolo Abeni

Hi Paolo,

Quoted 12 lines · Paolo Abeni
Quoted 11 lines · Matthieu Baerts

14 Aug 2026 10:43:41 Paolo Abeni <[email protected]>:

Quoted 2 lines

It's not clear to me what actually goes wrong and how. Do you have handy a event sequence or stack trace demonstrating the critical scenario?

Indeed, better with packetdrill scripts:

https://github.com/chrisocean716-star/packetdrill/blob/bf9be7a7bee3687500444b284a9db2a0009a5252/gtests/net/mptcp/dss/dss_drop_after_data_fallback_server.pkt

=> A fallback is done, a subflow can be initiated, then rejected.

Quoted 8 lines
Quoted 4 lines

https://github.com/chrisocean716-star/packetdrill/blob/fed57fab92c407eb433acdb8c8ce72bd8e3e2f37/gtests/net/mptcp/mp_join/mp_join_server_after_fallback.pkt

=> The MP_FAIL is partially ignored if received before being in fully established state.

Thanks!

I'm pretty sure there is a miss-intepretation of the 2nd case.

Thank you for having checked!

Quoted 4 lines

The problem is not with msk->fully_established. AFAICS msk->fully_established is true when the MP_FAIL is receive.

The passive msk socket sets it at MPC ACK reception.

Indeed, I should have checked the code instead of only reading the comments from this test :)

Quoted 9 lines

The problem is that, at MP_FAIL reception, the infinite mapping is not sent immediately, but it's scheduled with the next xmit, and until such point the msk does not perform the fallback.

I think a proper solution would be bounding MP_FAIL (reply) and infinite mapping in the same ack.

As noted by Gang, it may require a new helper (build on top of the tcp_send_ack).

Sounds good to me!

What about the 1st case (fallback)?

Would it help to reset msk->fully_established in __mptcp_try_fallback()?(while holding the data_lock? not even sure it is needed at that stage)
But here, we will also need to update subflow_token_join_request() to check mptcp_is_fully_established, or even mptcp_can_accept_new_subflow. (I can send a quick RFC patch)

It would also benefit to the in-kernel PM and others places using mptcp_is_fully_established(). subflow->fully_established doesn't need to be reset.

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
maintainer
#13 lore raw

Re: [PATCH export v3 2/4] mptcp: reject joins after fallback in mptcp_is_fully_established

Quoted 45 lines · Matthieu Baerts

Hi Paolo,

Quoted 12 lines · Paolo Abeni
Quoted 11 lines · Matthieu Baerts

14 Aug 2026 10:43:41 Paolo Abeni <[email protected]>:

Quoted 2 lines

It's not clear to me what actually goes wrong and how. Do you have handy a event sequence or stack trace demonstrating the critical scenario?

Indeed, better with packetdrill scripts:

https://github.com/chrisocean716-star/packetdrill/blob/bf9be7a7bee3687500444b284a9db2a0009a5252/gtests/net/mptcp/dss/dss_drop_after_data_fallback_server.pkt

=> A fallback is done, a subflow can be initiated, then rejected.

Quoted 8 lines
Quoted 4 lines

https://github.com/chrisocean716-star/packetdrill/blob/fed57fab92c407eb433acdb8c8ce72bd8e3e2f37/gtests/net/mptcp/mp_join/mp_join_server_after_fallback.pkt

=> The MP_FAIL is partially ignored if received before being in fully established state.

Thanks!

I'm pretty sure there is a miss-intepretation of the 2nd case.

Thank you for having checked!

Quoted 4 lines

The problem is not with msk->fully_established. AFAICS msk->fully_established is true when the MP_FAIL is receive.

The passive msk socket sets it at MPC ACK reception.

Indeed, I should have checked the code instead of only reading the comments from this test :)

Quoted 9 lines

The problem is that, at MP_FAIL reception, the infinite mapping is not sent immediately, but it's scheduled with the next xmit, and until such point the msk does not perform the fallback.

I think a proper solution would be bounding MP_FAIL (reply) and infinite mapping in the same ack.

As noted by Gang, it may require a new helper (build on top of the tcp_send_ack).

Sounds good to me!

Quoted 8 lines

What about the 1st case (fallback)?

Would it help to reset msk->fully_established in __mptcp_try_fallback()?(while holding the data_lock? not even sure it is needed at that stage)
But here, we will also need to update subflow_token_join_request() to check mptcp_is_fully_established, or even mptcp_can_accept_new_subflow. (I can send a quick RFC patch)

Sent:
https://lore.kernel.org/[email protected]

Quoted 6 lines

It would also benefit to the in-kernel PM and others places using mptcp_is_fully_established(). subflow->fully_established doesn't need to be reset.

Cheers,
Matt

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.