mptcp: fix add_addr_accepted accounting on subflow close
Subsystems (2)
PATCH 1/1 mptcp: fix add_addr_accepted accounting on subflow close
When an accepted remote address is used by a subflow that is closed before the corresponding RM_ADDR is received, the add_addr_accepted counter is not decremented when the subflow is removed.
As a result, the accepted-address budget can remain consumed for the lifetime of the MPTCP connection. A subsequent ADD_ADDR is then not acted upon because the peer is considered to have already reached the add_addr_accepted limit.
Track accepted remote address IDs in a bitmap instead of relying on a live subflow to signal removal. Release the accepted-address slot as soon as the subflow using it is closed, instead of depending on a matching RM_ADDR that may never arrive. The same bitmap also guards against double-counting if the same address ID is accepted more than once.
This allows a new MP_JOIN to be created after a previously accepted subflow has been closed and its address has been removed.
--- net/mptcp/pm.c | 8 +++--- net/mptcp/pm_kernel.c | 63 ++++++++++++++++++++++++++++++++++++------- net/mptcp/protocol.h | 3 +++ 3 files changed, 60 insertions(+), 14 deletions(-)
net/mptcp/pm.c +4 -4
@@ -681,8 +681,10 @@ void mptcp_pm_subflow_check_next(struct mptcp_sock *msk,return ;spin_lock_bh (&pm->lock); -if (update_subflows) +if (update_subflows) {__mptcp_pm_close_subflow (msk); +mptcp_pm_nl_close_subflow (msk, subflow); + }/* Even if this subflow is not really established, tell the PM to try * to pick the next ones, if possible.
@@ -786,7 +788,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,for (i =0 ; i < rm_list->nr; i++) {u8 rm_id = rm_list->ids[i]; -bool removed = false;mptcp_for_each_subflow_safe (msk, subflow, tmp) {struct sock *ssk =mptcp_subflow_tcp_sock (subflow);
@@ -807,7 +808,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk, i, rm_id, id, remote_id, msk->mpc_endpoint_id);spin_unlock_bh (&msk->pm.lock);mptcp_subflow_shutdown (sk, ssk, how); - removed |= subflow->request_join;/* the following takes care of updating the subflows counter */ mptcp_close_ssk (sk, ssk, subflow);
@@ -819,7 +819,7 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,if (rm_type ==MPTCP_MIB_RMADDR ) {__MPTCP_INC_STATS (sock_net (sk), rm_type); -if (removed &&mptcp_pm_is_kernel (msk)) +if (mptcp_pm_is_kernel (msk))mptcp_pm_nl_rm_addr (msk, rm_id); } }
net/mptcp/pm_kernel.c +53 -10
83 lines not on this page · raw message
net/mptcp/protocol.h +3 -0
@@ -243,6 +243,7 @@ struct mptcp_pm_data { );DECLARE_BITMAP (id_avail_bitmap,MPTCP_PM_MAX_ADDR_ID +1 ); +DECLARE_BITMAP (id_accepted_bitmap,MPTCP_PM_MAX_ADDR_ID +1 );struct mptcp_rm_list rm_list_tx;struct mptcp_rm_list rm_list_rx; };
@@ -1127,6 +1128,8 @@ void mptcp_pm_send_ack(struct mptcp_sock *msk, bool prio, bool backup);void mptcp_pm_addr_send_ack (struct mptcp_sock *msk);void mptcp_pm_nl_rm_addr (struct mptcp_sock *msk,u8 rm_id); +void mptcp_pm_nl_close_subflow (struct mptcp_sock *msk, +const struct mptcp_subflow_context *subflow);void mptcp_pm_rm_subflow (struct mptcp_sock *msk,const struct mptcp_rm_list *rm_list); void mptcp_pm_rm_addr_received(struct mptcp_sock *msk,
Signature · 1 line
2.43.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Concurrent overwrite of msk->pm.rm_list_rx while iterating over it in mptcp_pm_rm_addr_or_subflow. --
--- Patch [1]: [PATCH mptcp-next] mptcp: fix add_addr_accepted accounting on subflow close --- commit 66a438e0ebd8ec5a7b6eb2a506d61f6a2a2d2cc5
Author: Akshit Patadiya <[email protected]>
mptcp: fix add_addr_accepted accounting on subflow close
This commit modifies how MPTCP tracks accepted remote addresses by using a bitmap instead of relying on a live subflow to signal removal. It releases accepted-address slots as soon as the subflow is closed.
Quoted 4 lines
net/mptcp/pm.c +0
-0
[ ... ]
net/mptcp/pm.c static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk, · line 813
Quoted 8 lines
net/mptcp/pm.c +0
-1@@ -807,7 +808,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk, i, rm_id, id, remote_id, msk->mpc_endpoint_id);spin_unlock_bh (&msk->pm.lock);mptcp_subflow_shutdown (sk, ssk, how); - removed |= subflow->request_join;/* the following takes care of updating the subflows counter */ mptcp_close_ssk (sk, ssk, subflow);
[Severity: High]
This isn't a bug introduced by this patch, but could this unlock/lock sequence lead to a race condition with the outer loop?
In mptcp_pm_rm_addr_or_subflow(), the outer loop iterates over rm_list:
for (i = 0; i < rm_list->nr; i++) {
u8 rm_id = rm_list->ids[i];If rm_list points to &msk->pm.rm_list_rx, and the PM lock is dropped here in the inner loop, could a concurrent softirq processing a new RM_ADDR overwrite the list while the lock is temporarily released?
This might corrupt the ongoing iteration of the outer loop once the lock is reacquired and the next item in rm_list->nr and rm_list->ids is evaluated.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Hi Akshit,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/31703246069
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/a42cb09aedcb
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1145437
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-normalFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)
Quoted 6 lines · On Thu, Aug 13, 2026 at 05:19:09PM +0530, [email protected] wrote:
[High] Concurrent updates to
msk->pm.rm_list_rxcan causeRM_ADDRIDs to be silently skipped, resulting in leaked subflows.If a remote peer sends multiple
RM_ADDRoptions in rapid succession, softirq context can callmptcp_pm_rm_addr_received(), which unconditionally overwritespm->rm_list_rxwhile scheduling work
Looked into this, it's real. Traced it down:
mptcp_pm_rm_addr_recv() passes &msk->pm.rm_list_rx straight into mptcp_pm_rm_addr_or_subflow() (pm.c:830), and that loop re-reads rm_list->nr / rm_list->ids[i] on every iteration. It also drops pm.lock mid-loop around mptcp_subflow_shutdown()/mptcp_close_ssk() since those can't be called under a spinlock.
Worse, mptcp_pm_worker() clears the MPTCP_PM_RM_ADDR_RECEIVED bit before calling mptcp_pm_rm_addr_recv():
if (pm->status & BIT(MPTCP_PM_RM_ADDR_RECEIVED)) {
pm->status &= ~BIT(MPTCP_PM_RM_ADDR_RECEIVED);
mptcp_pm_rm_addr_recv(msk);
}so if a second RM_ADDR shows up while the first is still being processed, mptcp_pm_schedule_work() sees the bit already clear, treats it as new work, and mptcp_pm_rm_addr_received() overwrites rm_list_rx right under the loop during one of its unlock windows instead of getting coalesced. End result is the outer loop's i / rm_list->nr can go stale mid-flight and some IDs from the original batch just never get processed.
This is pre-existing, not something this patch touches (different struct, different lock concern than the id_accepted_bitmap stuff here), so I'm not folding a fix into this one. Will open a separate issue/patch for it, can take it myself if nobody's already on it.
For #498 itself, this patch should close it as-is. It also looks related to #496 per that issue's note - let me know if you want me to check whether this covers that one too, or if it's a prerequisite.
Tested against mptcp_join.sh for the close-before-RM_ADDR case, fullmesh reuse of the same remote id, and duplicate RM_ADDR - all passing.
Thanks and regards,
Akshit patadiya
From: Akshit Patadiya <[email protected]>
To: <[email protected]>
Cc: <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, "Akshit Patadiya"<[email protected]>
Date: Thu, 13 Aug 2026 18:07:54 +0530
Subject: [PATCH mptcp-next] mptcp: fix add_addr_accepted accounting on subflow close
Quoted 186 lines
When an accepted remote address is used by a subflow that is closed before the corresponding
RM_ADDRis received, theadd_addr_acceptedcounter is not decremented when the subflow is removed.As a result, the accepted-address budget can remain consumed for the lifetime of the MPTCP connection. A subsequent
ADD_ADDRis then not acted upon because the peer is considered to have already reached theadd_addr_acceptedlimit.Track accepted remote address IDs in a bitmap instead of relying on a live subflow to signal removal. Release the accepted-address slot as soon as the subflow using it is closed, instead of depending on a matching
RM_ADDRthat may never arrive. The same bitmap also guards against double-counting if the same address ID is accepted more than once.This allows a new
MP_JOINto be created after a previously accepted subflow has been closed and its address has been removed.Fixes 1c1f72137598 ("mptcp: pm: only decrement add_addr_accepted for MPJ req")Signed-off-by Akshit Patadiya <[email protected]>--- net/mptcp/pm.c | 8 +++--- net/mptcp/pm_kernel.c | 63 ++++++++++++++++++++++++++++++++++++------- net/mptcp/protocol.h | 3 +++ 3 files changed, 60 insertions(+), 14 deletions(-)net/mptcp/pm.c +4
-4@@ -681,8 +681,10 @@ void mptcp_pm_subflow_check_next(struct mptcp_sock *msk,return ;spin_lock_bh (&pm->lock); -if (update_subflows) +if (update_subflows) {__mptcp_pm_close_subflow (msk); +mptcp_pm_nl_close_subflow (msk, subflow); + }/* Even if this subflow is not really established, tell the PM to try * to pick the next ones, if possible. @@ -786,7 +788,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,for (i =0 ; i < rm_list->nr; i++) {u8 rm_id = rm_list->ids[i]; -bool removed = false;mptcp_for_each_subflow_safe (msk, subflow, tmp) {struct sock *ssk =mptcp_subflow_tcp_sock (subflow);@@ -807,7 +808,6 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk, i, rm_id, id, remote_id, msk->mpc_endpoint_id);spin_unlock_bh (&msk->pm.lock);mptcp_subflow_shutdown (sk, ssk, how); - removed |= subflow->request_join;/* the following takes care of updating the subflows counter */ mptcp_close_ssk (sk, ssk, subflow);@@ -819,7 +819,7 @@ static void mptcp_pm_rm_addr_or_subflow(struct mptcp_sock *msk,if (rm_type ==MPTCP_MIB_RMADDR ) {__MPTCP_INC_STATS (sock_net (sk), rm_type); -if (removed &&mptcp_pm_is_kernel (msk)) +if (mptcp_pm_is_kernel (msk))mptcp_pm_nl_rm_addr (msk, rm_id); } }net/mptcp/pm_kernel.c +53
-10@@ -694,8 +694,13 @@ static void mptcp_pm_nl_add_addr_received(struct mptcp_sock *msk)spin_lock_bh (&msk->pm.lock);if (sf_created) { -/* add_addr_accepted is not decr for ID 0 */ -if (remote.id) +/* ID 0 is not accounted: the remote address of the initial +* subflow is known from the beginning. Remember the other +* accepted IDs, so the counter can be balanced later on even +* if the linked subflows are gone by then. +*/ +if (remote.id && + !__test_and_set_bit (remote.id, msk->pm.id_accepted_bitmap)) msk->pm.add_addr_accepted++;if (msk->pm.add_addr_accepted >= limit_add_addr_accepted || msk->pm.extra_subflows >= limit_extra_subflows)@@ -705,16 +710,53 @@ static void mptcp_pm_nl_add_addr_received(struct mptcp_sock *msk)void mptcp_pm_nl_rm_addr (struct mptcp_sock *msk,u8 rm_id) { -if (rm_id && !WARN_ON_ONCE (msk->pm.add_addr_accepted ==0 )) { -u8 limit_add_addr_accepted = -mptcp_pm_get_limit_add_addr_accepted (msk); +u8 limit_add_addr_accepted; -/* Note: if the subflow has been closed before, this -* add_addr_accepted counter will not be decremented. -*/ -if (--msk->pm.add_addr_accepted < limit_add_addr_accepted) -WRITE_ONCE (msk->pm.accept_addr, true); +/* Only remote addresses that have been accepted by this host are +* accounted: not ID 0, and not MP_JOIN requests initiated by the peer. +* The bit, not the presence of a subflow, is what tells them apart, so +* this works even when the subflows are already closed, and a +* duplicated RM_ADDR is a no-op. +*/ +if (!rm_id || !__test_and_clear_bit (rm_id, msk->pm.id_accepted_bitmap)) +return ; + +if (WARN_ON_ONCE (msk->pm.add_addr_accepted ==0 )) +return ; + + limit_add_addr_accepted =mptcp_pm_get_limit_add_addr_accepted (msk); +if (--msk->pm.add_addr_accepted < limit_add_addr_accepted) +WRITE_ONCE (msk->pm.accept_addr, true); +} + +/* Called with the PM lock held, from the subflow close path, before the +* subflow is removed from conn_list. +*/ +void mptcp_pm_nl_close_subflow (struct mptcp_sock *msk, +const struct mptcp_subflow_context *subflow) +{ +u8 remote_id =READ_ONCE (subflow->remote_id); +struct mptcp_subflow_context *iter; + +/* Only the subflows this host has created upon an ADD_ADDR reception +* are accounted, and never the initial one. +*/ +if (!subflow->request_join || !remote_id || + !test_bit (remote_id, msk->pm.id_accepted_bitmap)) +return ; + +/* The remote address can still be used by another subflow, e.g. with +* fullmesh endpoints. +*/ +mptcp_for_each_subflow (msk, iter) { +if (iter == subflow || iter->close_event_done) +continue ; +if (iter->request_join && +READ_ONCE (iter->remote_id) == remote_id) +return ; } + +mptcp_pm_nl_rm_addr (msk, remote_id); }static bool address_use_port (struct mptcp_pm_addr_entry *entry)@@ -1668,6 +1710,7 @@ static void mptcp_pm_kernel_init(struct mptcp_sock *msk)WRITE_ONCE (pm->accept_subflow, subflows_allowed);bitmap_fill (pm->id_avail_bitmap,MPTCP_PM_MAX_ADDR_ID +1 ); +bitmap_zero (pm->id_accepted_bitmap,MPTCP_PM_MAX_ADDR_ID +1 ); }struct mptcp_pm_ops mptcp_pm_kernel = {net/mptcp/protocol.h +3
-0@@ -243,6 +243,7 @@ struct mptcp_pm_data { );DECLARE_BITMAP (id_avail_bitmap,MPTCP_PM_MAX_ADDR_ID +1 ); +DECLARE_BITMAP (id_accepted_bitmap,MPTCP_PM_MAX_ADDR_ID +1 );struct mptcp_rm_list rm_list_tx;struct mptcp_rm_list rm_list_rx; };@@ -1127,6 +1128,8 @@ void mptcp_pm_send_ack(struct mptcp_sock *msk, bool prio, bool backup);void mptcp_pm_addr_send_ack (struct mptcp_sock *msk);void mptcp_pm_nl_rm_addr (struct mptcp_sock *msk,u8 rm_id); +void mptcp_pm_nl_close_subflow (struct mptcp_sock *msk, +const struct mptcp_subflow_context *subflow);void mptcp_pm_rm_subflow (struct mptcp_sock *msk,const struct mptcp_rm_list *rm_list); void mptcp_pm_rm_addr_received(struct mptcp_sock *msk,Signature · 1 line
2.43.0
Akshit Patadiya
Team Lead & Software developer at Mpiric Software
View My Digital Business Card
Disclaimer: This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Mpiric Software.
Hi all,
Gentle ping on this patch. When you have a chance, could you please take a look and let me know if there are any further changes needed?
Quoted 6 lines · On Thu, Aug 13, 2026 at 05:19:09PM +0530, [email protected] wrote:
[High] Concurrent updates to
msk->pm.rm_list_rxcan causeRM_ADDRIDs to be silently skipped, resulting in leaked subflows.If a remote peer sends multiple
RM_ADDRoptions in rapid succession, softirq context can callmptcp_pm_rm_addr_received(), which unconditionally overwritespm->rm_list_rxwhile scheduling work
Looked into this, it's real. Traced it down:
mptcp_pm_rm_addr_recv() passes &msk->pm.rm_list_rx straight into mptcp_pm_rm_addr_or_subflow() (pm.c:830), and that loop re-reads rm_list->nr / rm_list->ids[i] on every iteration. It also drops pm.lock mid-loop around mptcp_subflow_shutdown()/mptcp_close_ssk() since those can't be called under a spinlock.
Worse, mptcp_pm_worker() clears the MPTCP_PM_RM_ADDR_RECEIVED bit before calling mptcp_pm_rm_addr_recv():
if (pm->status & BIT(MPTCP_PM_RM_ADDR_RECEIVED)) {
pm->status &= ~BIT(MPTCP_PM_RM_ADDR_RECEIVED);
mptcp_pm_rm_addr_recv(msk);
}so if a second RM_ADDR shows up while the first is still being processed, mptcp_pm_schedule_work() sees the bit already clear, treats it as new work, and mptcp_pm_rm_addr_received() overwrites rm_list_rx right under the loop during one of its unlock windows instead of getting coalesced. End result is the outer loop's i / rm_list->nr can go stale mid-flight and some IDs from the original batch just never get processed.
This is pre-existing, not something this patch touches (different struct, different lock concern than the id_accepted_bitmap stuff here), so I'm not folding a fix into this one. Will open a separate issue/patch for it, can take it myself if nobody's already on it.
For #498 itself, this patch should close it as-is. It also looks related to #496 per that issue's note - let me know if you want me to check whether this covers that one too, or if it's a prerequisite.
Tested against mptcp_join.sh for the close-before-RM_ADDR case, fullmesh reuse of the same remote id, and duplicate RM_ADDR - all passing.
Thanks and regards,
Akshit Patadiya