Skip to contentSkip to conversation

mptcp: fix race in mptcp_pm_nl_flush_addrs_doit()

PATCH In review Eric Dumazet 4 participants 6 messages Jan 23 – Jan 24

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mptcp: fix race in mptcp_pm_nl_flush_addrs_doit()

syzbot and Eulgyu Kim reported crashes in mptcp_pm_nl_get_local_id() and/or mptcp_pm_nl_is_backup()

Root cause is list_splice_init() in mptcp_pm_nl_flush_addrs_doit() which is not RCU ready.

list_splice_init_rcu() can not be called here while holding pernet->lock spinlock.

Many thanks to Eulgyu Kim for providing a repro and testing our patches.

Fixes 141694df6573 ("mptcp: remove address when netlink flushes addrs")
Reported-by [email protected]
Cc Geliang Tang <[email protected]>

---
v2: Make sure the list was not empty, return early otherwise.
v1: https://lore.kernel.org/netdev/[email protected]/

 net/mptcp/pm_kernel.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
net/mptcp/pm_kernel.c +0 -0
index 57570a44e4185370f531047fe97ce9f9fbd1480b..af23be6658ded4860133bb9495c7738014815d28 100644
@@ -1294,16 +1294,28 @@ static void __reset_counters(struct pm_nl_pernet *pernet)
 int mptcp_pm_nl_flush_addrs_doit(struct sk_buff *skb, struct genl_info *info)
 {
	struct pm_nl_pernet *pernet = genl_info_pm_nl(info);

- LIST_HEAD(free_list);
+ struct list_head free_list;

	spin_lock_bh(&pernet->lock);

- list_splice_init(&pernet->endp_list, &free_list);
+
+ free_list = pernet->endp_list;
+ INIT_LIST_HEAD_RCU(&pernet->endp_list);
+

	__reset_counters(pernet);
	pernet->next_id = 1;
	bitmap_zero(pernet->id_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
	spin_unlock_bh(&pernet->lock);

- mptcp_nl_flush_addrs_list(sock_net(skb->sk), &free_list);
+
+ if (free_list.next == &pernet->endp_list)
+ return 0;
+

	synchronize_rcu();

+
+ /* Adjust the pointers to free_list instead of pernet->endp_list */
+ free_list.prev->next = &free_list;
+ free_list.next->prev = &free_list;
+

+	mptcp_nl_flush_addrs_list(sock_net(skb->sk), &free_list);
	__flush_addrs(&free_list);
	return 0;

}
--
2.52.0.457.g6b5491de43-goog

#1 lore raw

Hi Eric,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Unstable: 1 failed test(s): selftest_simult_flows 🔴
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/21281916251

Initiator: Matthieu Baerts (NGI0)
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/c14ce3016a03
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1045988

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

#2 lore raw
Quoted 33 lines · MPTCP CI

Hi Eric,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Unstable: 1 failed test(s): selftest_simult_flows 🔴
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/21281916251

Initiator: Matthieu Baerts (NGI0)
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/c14ce3016a03
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1045988

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Quoted 8 lines

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

This seem a flake ?

No issue here

make -j8 -C tools/testing/selftests TARGETS=net/mptcp TEST_PROGS="simult_flows.sh" run_tests
make: Entering directory '/usr/local/google/home/edumazet/git/net-next/tools/testing/selftests' make[1]: Nothing to be done for 'all'.
TAP version 13
1..1

# timeout set to 1800
# selftests: net/mptcp: simult_flows.sh
# 01 balanced bwidth                                             7412
max 7906       [ OK ]
# 02 balanced bwidth - reverse direction                         7389
max 7906       [ OK ]
# 03 balanced bwidth with unbalanced delay                       7522
max 7906       [ OK ]
# 04 balanced bwidth with unbalanced delay - reverse direction   7519
max 7906       [ OK ]
# 05 unbalanced bwidth                                           11588
max 11921      [ OK ]
# 06 unbalanced bwidth - reverse direction                       11431
max 11921      [ OK ]
# 07 unbalanced bwidth with unbalanced delay                     11378
max 11921      [ OK ]
# 08 unbalanced bwidth with unbalanced delay - reverse direction 11304
max 11921      [ OK ]
# 09 unbalanced bwidth with opposed, unbalanced delay            11605
max 11921      [ OK ]
# 10 unbalanced bwidth with opposed, unbalanced delay - reverse
direction11579 max 11921      [ OK ]
#
# TAP version 13
# 1..10
# ok 1 - simult_flows: balanced bwidth # time=9708ms
# ok 2 - simult_flows: balanced bwidth - reverse direction # time=8764ms
# ok 3 - simult_flows: balanced bwidth with unbalanced delay # time=9414ms
# ok 4 - simult_flows: balanced bwidth with unbalanced delay - reverse
direction # time=8847ms
# ok 5 - simult_flows: unbalanced bwidth # time=13405ms
# ok 6 - simult_flows: unbalanced bwidth - reverse direction # time=12701ms
# ok 7 - simult_flows: unbalanced bwidth with unbalanced delay # time=13247ms
# ok 8 - simult_flows: unbalanced bwidth with unbalanced delay -
reverse direction # time=12589ms
# ok 9 - simult_flows: unbalanced bwidth with opposed, unbalanced
delay # time=13441ms
# ok 10 - simult_flows: unbalanced bwidth with opposed, unbalanced
delay - reverse direction # time=12940ms
ok 1 selftests: net/mptcp: simult_flows.sh
make: Leaving directory
'/usr/local/google/home/edumazet/git/net-next/tools/testing/selftests'
root@virtme-ng:/usr/local/google/home/edumazet/git/net-next#
maintainer
#3 lore raw

Hi Eric,

Thank you for having looked at this and sharing this fix.

23 Jan 2026 12:36:37 Eric Dumazet <[email protected]>:

Quoted 8 lines
Quoted 3 lines · MPTCP CI

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Unstable: 1 failed test(s): selftest_simult_flows 🔴

This seem a flake ?

Yes it is, and due to a commit ("mptcp: better mptcp-level RTT estimator") which is only in our tree (used by our CI), see:

https://github.com/multipath-tcp/mptcp_net-next/issues/607

Sorry, I didn't have the opportunity to find a solution for that.

Regarding your patch, it looks like it is corrupted. Both NIPA and Patchew are complaining about it:

  Applying: mptcp: fix race in mptcp_pm_nl_flush_addrs_doit()
  error: corrupt patch at line 15
  error: could not build fake ancestor

"git am" complains, but it is OK to manually apply it with "patch". Any idea why? If you are not available, do you want me to resend it later on?

Cheers,
Matt

maintainer Reviewed-by on v2 1/1
#4 lore raw
In reply to Eric Dumazet
Quoted 11 lines · Eric Dumazet

syzbot and Eulgyu Kim reported crashes in mptcp_pm_nl_get_local_id() and/or mptcp_pm_nl_is_backup()

Root cause is list_splice_init() in mptcp_pm_nl_flush_addrs_doit() which is not RCU ready.

list_splice_init_rcu() can not be called here while holding pernet->lock spinlock.

Many thanks to Eulgyu Kim for providing a repro and testing our patches.

Quoted 8 lines
Fixes 141694df6573 ("mptcp: remove address when netlink flushes addrs")
Reported-by [email protected]
Cc Geliang Tang <[email protected]>

---
v2: Make sure the list was not empty, return early otherwise.

Thanks Eric, the v2 code changes LGTM. The netdev tooling wasn't able to apply the patch (https://patchwork.kernel.org/project/netdevbpf/patch/[email protected]/), so Matthieu is planning to send a basically-identical v3 that 'git am' and the netdev CI will be happy with.

Quoted 45 lines

v1: https://lore.kernel.org/netdev/[email protected]/

net/mptcp/pm_kernel.c | 18 +++++++++++++++---

1 file changed, 15 insertions(+), 3 deletions(-)
net/mptcp/pm_kernel.c +15 -4
index 57570a44e4185370f531047fe97ce9f9fbd1480b..af23be6658ded4860133bb9495c7738014815d28 100644
@@ -1294,16 +1294,28 @@ static void __reset_counters(struct pm_nl_pernet *pernet)
 int mptcp_pm_nl_flush_addrs_doit(struct sk_buff *skb, struct genl_info *info)
 {
 	struct pm_nl_pernet *pernet = genl_info_pm_nl(info);
-	LIST_HEAD(free_list);
+	struct list_head free_list;
 
 	spin_lock_bh(&pernet->lock);
-	list_splice_init(&pernet->endp_list, &free_list);
+
+	free_list = pernet->endp_list;
+	INIT_LIST_HEAD_RCU(&pernet->endp_list);
+
 	__reset_counters(pernet);
 	pernet->next_id = 1;
 	bitmap_zero(pernet->id_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
 	spin_unlock_bh(&pernet->lock);
-	mptcp_nl_flush_addrs_list(sock_net(skb->sk), &free_list);
+
+	if (free_list.next == &pernet->endp_list)
+		return 0;
+
 	synchronize_rcu();
+
+	/* Adjust the pointers to free_list instead of pernet->endp_list */
+	free_list.prev->next = &free_list;
+	free_list.next->prev = &free_list;
+
+	mptcp_nl_flush_addrs_list(sock_net(skb->sk), &free_list);
 	__flush_addrs(&free_list);
 	return 0;
 }
--

2.52.0.457.g6b5491de43-goog

maintainer
#5 lore raw

Hi Mat, Eric,

Quoted 23 lines · Mat Martineau
Quoted 15 lines · Eric Dumazet

syzbot and Eulgyu Kim reported crashes in mptcp_pm_nl_get_local_id() and/or mptcp_pm_nl_is_backup()

Root cause is list_splice_init() in mptcp_pm_nl_flush_addrs_doit() which is not RCU ready.

list_splice_init_rcu() can not be called here while holding pernet->lock spinlock.

Many thanks to Eulgyu Kim for providing a repro and testing our patches.

Fixes 141694df6573 ("mptcp: remove address when netlink flushes addrs")
Reported-by [email protected]
Quoted 5 lines

all/[email protected]/T/

---
v2: Make sure the list was not empty, return early otherwise.

Quoted 1 line

Thanks Eric, the v2 code changes LGTM.

Thank you for the patch and the review!

Quoted 5 lines

The netdev tooling wasn't able to
apply the patch (https://patchwork.kernel.org/project/netdevbpf/
patch/[email protected]/), so Matthieu is planning to send a basically-identical v3 that 'git am' and the netdev CI will be happy with.

Just did:

https://lore.kernel.org/20260124-net-mptcp-race_nl_flush_addrs-v3-1-b2dc1b613e9d@kernel.org

I guess we need to manually update Patchwork if the sender is different:

pw-bot: superseded

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.