Skip to contentSkip to conversation

mptcp: fix add_addr_accepted accounting on subflow close

PATCH In review Akshit Patadiya 3 participants 5 messages Aug 13 – Sep 15

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mptcp: fix add_addr_accepted accounting on subflow close

When an accepted remote address is used by a subflow that is closed before the corresponding RM_ADDR is received, the add_addr_accepted counter is not decremented when the subflow is removed.

As a result, the accepted-address budget can remain consumed for the lifetime of the MPTCP connection. A subsequent ADD_ADDR is then not acted upon because the peer is considered to have already reached the add_addr_accepted limit.

Track accepted remote address IDs in a bitmap instead of relying on a live subflow to signal removal. Release the accepted-address slot as soon as the subflow using it is closed, instead of depending on a matching RM_ADDR that may never arrive. The same bitmap also guards against double-counting if the same address ID is accepted more than once.

This allows a new MP_JOIN to be created after a previously accepted subflow has been closed and its address has been removed.

Fixes 1c1f72137598 ("mptcp: pm: only decrement add_addr_accepted for MPJ req")
---
 net/mptcp/pm.c        |  8 +++---
 net/mptcp/pm_kernel.c | 63 ++++++++++++++++++++++++++++++++++++-------
 net/mptcp/protocol.h  |  3 +++
 3 files changed, 60 insertions(+), 14 deletions(-)
net/mptcp/pm.c +4 -4
index ba7c6f80a183..b6edb9df3216 100644
@@ -681,8 +681,10 @@ void mptcp_pm_subflow_check_next(struct mptcp_sock *msk,
 		return;
 
 	spin_lock_bh(&pm->lock);
-	if (update_subflows)
+	if (update_subflows) {
 		__mptcp_pm_close_subflow(msk);
+		mptcp_pm_nl_close_subflow(msk, subflow);
+	}
 
 	/* Even if this subflow is not really established, tell the PM to try
 	 * to pick the next ones, if possible.
@@ -786,7 +788,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,
 
 	for (i = 0; i < rm_list->nr; i++) {
 		u8 rm_id = rm_list->ids[i];
-		bool removed = false;
 
 		mptcp_for_each_subflow_safe(msk, subflow, tmp) {
 			struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
@@ -807,7 +808,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,
 				 i, rm_id, id, remote_id, msk->mpc_endpoint_id);
 			spin_unlock_bh(&msk->pm.lock);
 			mptcp_subflow_shutdown(sk, ssk, how);
-			removed |= subflow->request_join;
 
 			/* the following takes care of updating the subflows counter */
 			mptcp_close_ssk(sk, ssk, subflow);
@@ -819,7 +819,7 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,
 
 		if (rm_type == MPTCP_MIB_RMADDR) {
 			__MPTCP_INC_STATS(sock_net(sk), rm_type);
-			if (removed && mptcp_pm_is_kernel(msk))
+			if (mptcp_pm_is_kernel(msk))
 				mptcp_pm_nl_rm_addr(msk, rm_id);
 		}
 	}
net/mptcp/pm_kernel.c +53 -10

83 lines not on this page · raw message

net/mptcp/protocol.h +3 -0
index 7e168e450fb0..54663d7aca34 100644
@@ -243,6 +243,7 @@ struct mptcp_pm_data {
 	);
 
 	DECLARE_BITMAP(id_avail_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
+	DECLARE_BITMAP(id_accepted_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
 	struct mptcp_rm_list rm_list_tx;
 	struct mptcp_rm_list rm_list_rx;
 };
@@ -1127,6 +1128,8 @@ void mptcp_pm_send_ack(struct mptcp_sock *msk,
 		       bool prio, bool backup);
 void mptcp_pm_addr_send_ack(struct mptcp_sock *msk);
 void mptcp_pm_nl_rm_addr(struct mptcp_sock *msk, u8 rm_id);
+void mptcp_pm_nl_close_subflow(struct mptcp_sock *msk,
+			       const struct mptcp_subflow_context *subflow);
 void mptcp_pm_rm_subflow(struct mptcp_sock *msk,
 			 const struct mptcp_rm_list *rm_list);
 void mptcp_pm_rm_addr_received(struct mptcp_sock *msk,
Signature · 1 line
2.43.0
#1 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Concurrent overwrite of msk->pm.rm_list_rx while iterating over it in mptcp_pm_rm_addr_or_subflow. --

--- Patch [1]: [PATCH mptcp-next] mptcp: fix add_addr_accepted accounting on subflow close --- commit 66a438e0ebd8ec5a7b6eb2a506d61f6a2a2d2cc5
Author: Akshit Patadiya <[email protected]>

mptcp: fix add_addr_accepted accounting on subflow close

This commit modifies how MPTCP tracks accepted remote addresses by using a bitmap instead of relying on a live subflow to signal removal. It releases accepted-address slots as soon as the subflow is closed.

Quoted 4 lines
net/mptcp/pm.c +0 -0
index ba7c6f80a183c..b6edb9df3216c 100644

[ ... ]

net/mptcp/pm.c static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk, · line 813

Quoted 8 lines
net/mptcp/pm.c +0 -1
@@ -807,7 +808,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,
 				 i, rm_id, id, remote_id, msk->mpc_endpoint_id);
 			spin_unlock_bh(&msk->pm.lock);
 			mptcp_subflow_shutdown(sk, ssk, how);
-			removed |= subflow->request_join;
 
 			/* the following takes care of updating the subflows counter */
 			mptcp_close_ssk(sk, ssk, subflow);

[Severity: High]
This isn't a bug introduced by this patch, but could this unlock/lock sequence lead to a race condition with the outer loop?

In mptcp_pm_rm_addr_or_subflow(), the outer loop iterates over rm_list:

	for (i = 0; i < rm_list->nr; i++) {
		u8 rm_id = rm_list->ids[i];

If rm_list points to &msk->pm.rm_list_rx, and the PM lock is dropped here in the inner loop, could a concurrent softirq processing a new RM_ADDR overwrite the list while the lock is temporarily released?

This might corrupt the ongoing iteration of the outer loop once the lock is reacquired and the next item in rm_list->nr and rm_list->ids is evaluated.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#2 lore raw
In reply to Akshit Patadiya

Hi Akshit,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/31703246069

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/a42cb09aedcb
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1145437

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

#3 lore raw
In reply to Akshit Patadiya
Quoted 6 lines · On Thu, Aug 13, 2026 at 05:19:09PM +0530, [email protected] wrote:

[High] Concurrent updates to msk->pm.rm_list_rx can cause RM_ADDR IDs to be silently skipped, resulting in leaked subflows.

If a remote peer sends multiple RM_ADDR options in rapid succession, softirq context can call mptcp_pm_rm_addr_received(), which unconditionally overwrites pm->rm_list_rx while scheduling work

Looked into this, it's real. Traced it down:

mptcp_pm_rm_addr_recv() passes &msk->pm.rm_list_rx straight into mptcp_pm_rm_addr_or_subflow() (pm.c:830), and that loop re-reads rm_list->nr / rm_list->ids[i] on every iteration. It also drops pm.lock mid-loop around mptcp_subflow_shutdown()/mptcp_close_ssk() since those can't be called under a spinlock.

Worse, mptcp_pm_worker() clears the MPTCP_PM_RM_ADDR_RECEIVED bit before calling mptcp_pm_rm_addr_recv():

	if (pm->status & BIT(MPTCP_PM_RM_ADDR_RECEIVED)) {
		pm->status &= ~BIT(MPTCP_PM_RM_ADDR_RECEIVED);
		mptcp_pm_rm_addr_recv(msk);
	}

so if a second RM_ADDR shows up while the first is still being processed, mptcp_pm_schedule_work() sees the bit already clear, treats it as new work, and mptcp_pm_rm_addr_received() overwrites rm_list_rx right under the loop during one of its unlock windows instead of getting coalesced. End result is the outer loop's i / rm_list->nr can go stale mid-flight and some IDs from the original batch just never get processed.

This is pre-existing, not something this patch touches (different struct, different lock concern than the id_accepted_bitmap stuff here), so I'm not folding a fix into this one. Will open a separate issue/patch for it, can take it myself if nobody's already on it.

For #498 itself, this patch should close it as-is. It also looks related to #496 per that issue's note - let me know if you want me to check whether this covers that one too, or if it's a prerequisite.

Tested against mptcp_join.sh for the close-before-RM_ADDR case, fullmesh reuse of the same remote id, and duplicate RM_ADDR - all passing.

Thanks and regards,
Akshit patadiya

From: Akshit Patadiya <[email protected]>
To: <[email protected]>
Cc: <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, "Akshit Patadiya"<[email protected]>
Date: Thu, 13 Aug 2026 18:07:54 +0530
Subject: [PATCH mptcp-next] mptcp: fix add_addr_accepted accounting on subflow close

Quoted 186 lines

When an accepted remote address is used by a subflow that is closed before the corresponding RM_ADDR is received, the add_addr_accepted counter is not decremented when the subflow is removed.

As a result, the accepted-address budget can remain consumed for the lifetime of the MPTCP connection. A subsequent ADD_ADDR is then not acted upon because the peer is considered to have already reached the add_addr_accepted limit.

Track accepted remote address IDs in a bitmap instead of relying on a live subflow to signal removal. Release the accepted-address slot as soon as the subflow using it is closed, instead of depending on a matching RM_ADDR that may never arrive. The same bitmap also guards against double-counting if the same address ID is accepted more than once.

This allows a new MP_JOIN to be created after a previously accepted subflow has been closed and its address has been removed.

Fixes 1c1f72137598 ("mptcp: pm: only decrement add_addr_accepted for MPJ req")
---
 net/mptcp/pm.c        |  8 +++---
 net/mptcp/pm_kernel.c | 63 ++++++++++++++++++++++++++++++++++++-------
 net/mptcp/protocol.h  |  3 +++
 3 files changed, 60 insertions(+), 14 deletions(-)
net/mptcp/pm.c +4 -4
index ba7c6f80a183..b6edb9df3216 100644
@@ -681,8 +681,10 @@ void mptcp_pm_subflow_check_next(struct mptcp_sock *msk,
         return;
 
     spin_lock_bh(&pm->lock);
-    if (update_subflows)
+    if (update_subflows) {
         __mptcp_pm_close_subflow(msk);
+        mptcp_pm_nl_close_subflow(msk, subflow);
+    }
 
     /* Even if this subflow is not really established, tell the PM to try
      * to pick the next ones, if possible.
@@ -786,7 +788,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,
 
     for (i = 0; i < rm_list->nr; i++) {
         u8 rm_id = rm_list->ids[i];
-        bool removed = false;
 
         mptcp_for_each_subflow_safe(msk, subflow, tmp) {
             struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
@@ -807,7 +808,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,
                  i, rm_id, id, remote_id, msk->mpc_endpoint_id);
             spin_unlock_bh(&msk->pm.lock);
             mptcp_subflow_shutdown(sk, ssk, how);
-            removed |= subflow->request_join;
 
             /* the following takes care of updating the subflows counter */
             mptcp_close_ssk(sk, ssk, subflow);
@@ -819,7 +819,7 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,
 
         if (rm_type == MPTCP_MIB_RMADDR) {
             __MPTCP_INC_STATS(sock_net(sk), rm_type);
-            if (removed && mptcp_pm_is_kernel(msk))
+            if (mptcp_pm_is_kernel(msk))
                 mptcp_pm_nl_rm_addr(msk, rm_id);
         }
     }
net/mptcp/pm_kernel.c +53 -10
index d3014bf57bf3..8b087677a4e7 100644
@@ -694,8 +694,13 @@ static void mptcp_pm_nl_add_addr_received(struct mptcp_sock *msk)
     spin_lock_bh(&msk->pm.lock);
 
     if (sf_created) {
-        /* add_addr_accepted is not decr for ID 0 */
-        if (remote.id)
+        /* ID 0 is not accounted: the remote address of the initial
+         * subflow is known from the beginning. Remember the other
+         * accepted IDs, so the counter can be balanced later on even
+         * if the linked subflows are gone by then.
+         */
+        if (remote.id &&
+            !__test_and_set_bit(remote.id, msk->pm.id_accepted_bitmap))
             msk->pm.add_addr_accepted++;
         if (msk->pm.add_addr_accepted >= limit_add_addr_accepted ||
             msk->pm.extra_subflows >= limit_extra_subflows)
@@ -705,16 +710,53 @@ static void mptcp_pm_nl_add_addr_received(struct mptcp_sock *msk)
 
 void mptcp_pm_nl_rm_addr(struct mptcp_sock *msk, u8 rm_id)
 {
-    if (rm_id && !WARN_ON_ONCE(msk->pm.add_addr_accepted == 0)) {
-        u8 limit_add_addr_accepted =
-            mptcp_pm_get_limit_add_addr_accepted(msk);
+    u8 limit_add_addr_accepted;
 
-        /* Note: if the subflow has been closed before, this
-         * add_addr_accepted counter will not be decremented.
-         */
-        if (--msk->pm.add_addr_accepted < limit_add_addr_accepted)
-            WRITE_ONCE(msk->pm.accept_addr, true);
+    /* Only remote addresses that have been accepted by this host are
+     * accounted: not ID 0, and not MP_JOIN requests initiated by the peer.
+     * The bit, not the presence of a subflow, is what tells them apart, so
+     * this works even when the subflows are already closed, and a
+     * duplicated RM_ADDR is a no-op.
+     */
+    if (!rm_id || !__test_and_clear_bit(rm_id, msk->pm.id_accepted_bitmap))
+        return;
+
+    if (WARN_ON_ONCE(msk->pm.add_addr_accepted == 0))
+        return;
+
+    limit_add_addr_accepted = mptcp_pm_get_limit_add_addr_accepted(msk);
+    if (--msk->pm.add_addr_accepted < limit_add_addr_accepted)
+        WRITE_ONCE(msk->pm.accept_addr, true);
+}
+
+/* Called with the PM lock held, from the subflow close path, before the
+ * subflow is removed from conn_list.
+ */
+void mptcp_pm_nl_close_subflow(struct mptcp_sock *msk,
+                   const struct mptcp_subflow_context *subflow)
+{
+    u8 remote_id = READ_ONCE(subflow->remote_id);
+    struct mptcp_subflow_context *iter;
+
+    /* Only the subflows this host has created upon an ADD_ADDR reception
+     * are accounted, and never the initial one.
+     */
+    if (!subflow->request_join || !remote_id ||
+        !test_bit(remote_id, msk->pm.id_accepted_bitmap))
+        return;
+
+    /* The remote address can still be used by another subflow, e.g. with
+     * fullmesh endpoints.
+     */
+    mptcp_for_each_subflow(msk, iter) {
+        if (iter == subflow || iter->close_event_done)
+            continue;
+        if (iter->request_join &&
+            READ_ONCE(iter->remote_id) == remote_id)
+            return;
     }
+
+    mptcp_pm_nl_rm_addr(msk, remote_id);
 }
 
 static bool address_use_port(struct mptcp_pm_addr_entry *entry)
@@ -1668,6 +1710,7 @@ static void mptcp_pm_kernel_init(struct mptcp_sock *msk)
     WRITE_ONCE(pm->accept_subflow, subflows_allowed);
 
     bitmap_fill(pm->id_avail_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
+    bitmap_zero(pm->id_accepted_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
 }
 
 struct mptcp_pm_ops mptcp_pm_kernel = {
net/mptcp/protocol.h +3 -0
index 7e168e450fb0..54663d7aca34 100644
@@ -243,6 +243,7 @@ struct mptcp_pm_data {
     );
 
     DECLARE_BITMAP(id_avail_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
+    DECLARE_BITMAP(id_accepted_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
     struct mptcp_rm_list rm_list_tx;
     struct mptcp_rm_list rm_list_rx;
 };
@@ -1127,6 +1128,8 @@ void mptcp_pm_send_ack(struct mptcp_sock *msk,
                bool prio, bool backup);
 void mptcp_pm_addr_send_ack(struct mptcp_sock *msk);
 void mptcp_pm_nl_rm_addr(struct mptcp_sock *msk, u8 rm_id);
+void mptcp_pm_nl_close_subflow(struct mptcp_sock *msk,
+                   const struct mptcp_subflow_context *subflow);
 void mptcp_pm_rm_subflow(struct mptcp_sock *msk,
              const struct mptcp_rm_list *rm_list);
 void mptcp_pm_rm_addr_received(struct mptcp_sock *msk,
Signature · 1 line
2.43.0

Akshit Patadiya

Team Lead & Software developer at Mpiric Software

View My Digital Business Card

Disclaimer: This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Mpiric Software.

#4 lore raw
In reply to Akshit Patadiya

Hi all,

Gentle ping on this patch. When you have a chance, could you please take a look and let me know if there are any further changes needed?

Quoted 6 lines · On Thu, Aug 13, 2026 at 05:19:09PM +0530, [email protected] wrote:

[High] Concurrent updates to msk->pm.rm_list_rx can cause RM_ADDR IDs to be silently skipped, resulting in leaked subflows.

If a remote peer sends multiple RM_ADDR options in rapid succession, softirq context can call mptcp_pm_rm_addr_received(), which unconditionally overwrites pm->rm_list_rx while scheduling work

Looked into this, it's real. Traced it down:

mptcp_pm_rm_addr_recv() passes &msk->pm.rm_list_rx straight into mptcp_pm_rm_addr_or_subflow() (pm.c:830), and that loop re-reads rm_list->nr / rm_list->ids[i] on every iteration. It also drops pm.lock mid-loop around mptcp_subflow_shutdown()/mptcp_close_ssk() since those can't be called under a spinlock.

Worse, mptcp_pm_worker() clears the MPTCP_PM_RM_ADDR_RECEIVED bit before calling mptcp_pm_rm_addr_recv():

	if (pm->status & BIT(MPTCP_PM_RM_ADDR_RECEIVED)) {
		pm->status &= ~BIT(MPTCP_PM_RM_ADDR_RECEIVED);
		mptcp_pm_rm_addr_recv(msk);
	}

so if a second RM_ADDR shows up while the first is still being processed, mptcp_pm_schedule_work() sees the bit already clear, treats it as new work, and mptcp_pm_rm_addr_received() overwrites rm_list_rx right under the loop during one of its unlock windows instead of getting coalesced. End result is the outer loop's i / rm_list->nr can go stale mid-flight and some IDs from the original batch just never get processed.

This is pre-existing, not something this patch touches (different struct, different lock concern than the id_accepted_bitmap stuff here), so I'm not folding a fix into this one. Will open a separate issue/patch for it, can take it myself if nobody's already on it.

For #498 itself, this patch should close it as-is. It also looks related to #496 per that issue's note - let me know if you want me to check whether this covers that one too, or if it's a prerequisite.

Tested against mptcp_join.sh for the close-before-RM_ADDR case, fullmesh reuse of the same remote id, and duplicate RM_ADDR - all passing.

Thanks and regards,
Akshit Patadiya