Skip to contentSkip to conversation

only remove entry from local_addr_list when sending a REMOVE_ADDR

PATCH In review Geliang Tang 3 participants 8 messages Jun 26 – Jul 29

Subsystems (3)
lore
Share & tools
lore

Geliang Tang

author reviewer
#0 lore raw

v6 cover only remove entry from local_addr_list when sending a REMOVE_ADDR

From: Geliang Tang <[email protected]>

v6:
- patch 2: now explicitly removes userspace pm addresses using userspace_pm_rm_addr, instead of merely updating dump_addr expectations.
- patch 3: dropped duplicate "sleep 0.5".
- The pre-existing issue Sashiko mentioned in v5, "it's not safe to decrement sk->sk_omem_alloc synchronously while deferring the actual memory free via kfree_rcu_mightsleep(match)", will be addressed in a separate patch.

v5:
- rebased.
- add Matt's reviewed tags.
- add a RM_ADDR from the server to the client as Sashiko suggested.
- mptcpd tests (make check) are still OK after this modification.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v4:
- rebased.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v3:
- update userspace selftests.
- Link: https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

v2:
- squash patch 1 into patch 2 as Mat suggested.

Geliang Tang (3):
mptcp: pm: userspace: drop delete_local_addr helper
selftests: mptcp: join: explicitly remove userspace pm addresses
selftests: mptcp: userspace: send RM_ADDR between server and client

 net/mptcp/pm_userspace.c                      | 35 ++-----------------
 .../testing/selftests/net/mptcp/mptcp_join.sh |  4 ++-
 .../selftests/net/mptcp/userspace_pm.sh       | 14 ++++++++
 3 files changed, 19 insertions(+), 34 deletions(-)
Signature · 1 line
2.53.0
author reviewer
#1 lore raw

PATCH v6 1/3 mptcp: pm: userspace: drop delete_local_addr helper

From: Geliang Tang <[email protected]>

Address entries should not be removed from local_addr_list when a subflow is deleted by the userspace PM, should only be removed when sending a REMOVE_ADDR.

So mptcp_userspace_pm_delete_local_addr() helper shouldn't be called in mptcp_pm_nl_subflow_create_doit() and mptcp_pm_nl_subflow_destroy_doit().

Since this helper is open-coding in mptcp_pm_nl_remove_doit(), it can be dropped now.

Address entries are removed from local_addr_list when sending a REMOVE_ADDR by the userspace PM, the local_addr_used counter of PM should also be decremented accordingly.

---
 net/mptcp/pm_userspace.c | 35 ++---------------------------------
 1 file changed, 2 insertions(+), 33 deletions(-)
net/mptcp/pm_userspace.c +2 -33
index ad6ba658e5a5..45463f2f1e60 100644
@@ -90,30 +90,6 @@ static int mptcp_userspace_pm_append_new_local_addr(struct mptcp_sock *msk,
 	return ret;
 }
 
-/* If the subflow is closed from the other peer (not via a
- * subflow destroy command then), we want to keep the entry
- * not to assign the same ID to another address and to be
- * able to send RM_ADDR after the removal of the subflow.
- */
-static int mptcp_userspace_pm_delete_local_addr(struct mptcp_sock *msk,
-						struct mptcp_pm_addr_entry *addr)
-{
-	struct sock *sk = (struct sock *)msk;
-	struct mptcp_pm_addr_entry *entry;
-
-	entry = mptcp_userspace_pm_lookup_addr(msk, &addr->addr);
-	if (!entry)
-		return -EINVAL;
-
-	/* TODO: a refcount is needed because the entry can
-	 * be used multiple times (e.g. fullmesh mode).
-	 */
-	list_del_rcu(&entry->list);
-	sock_kfree_s(sk, entry, sizeof(*entry));
-	msk->pm.local_addr_used--;
-	return 0;
-}
-
 static struct mptcp_pm_addr_entry *
 mptcp_userspace_pm_lookup_addr_by_id(struct mptcp_sock *msk, unsigned int id)
 {
@@ -331,6 +307,7 @@ int mptcp_pm_nl_remove_doit(struct sk_buff *skb, struct genl_info *info)
 	}
 
 	list_del_rcu(&match->list);
+	msk->pm.local_addr_used--;
 	spin_unlock_bh(&msk->pm.lock);
 
 	mptcp_pm_remove_addr_entry(msk, match);
@@ -416,14 +393,9 @@ int mptcp_pm_nl_subflow_create_doit(struct sk_buff *skb, struct genl_info *info)
 	err = __mptcp_subflow_connect(sk, &local, &addr_r);
 	release_sock(sk);
 
-	if (err) {
+	if (err)
 		GENL_SET_ERR_MSG_FMT(info, "connect error: %d", err);
 
-		spin_lock_bh(&msk->pm.lock);
-		mptcp_userspace_pm_delete_local_addr(msk, &entry);
-		spin_unlock_bh(&msk->pm.lock);
-	}
-
  create_err:
 	sock_put(sk);
 	return err;
@@ -540,9 +512,6 @@ int mptcp_pm_nl_subflow_destroy_doit(struct sk_buff *skb, struct genl_info *info
 		goto release_sock;
 	}
 
-	spin_lock_bh(&msk->pm.lock);
-	mptcp_userspace_pm_delete_local_addr(msk, &addr_l);
-	spin_unlock_bh(&msk->pm.lock);
 	mptcp_subflow_shutdown(sk, ssk, RCV_SHUTDOWN | SEND_SHUTDOWN);
 	mptcp_close_ssk(sk, ssk, mptcp_subflow_ctx(ssk));
 	MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RMSUBFLOW);
Signature · 1 line
2.53.0
maintainer
#2 lore raw

Re: [PATCH mptcp-next v6 1/3] mptcp: pm: userspace: drop delete_local_addr helper

Hi Geliang,

Quoted 9 lines · Geliang Tang

From: Geliang Tang <[email protected]>

Address entries should not be removed from local_addr_list when a subflow is deleted by the userspace PM, should only be removed when sending a REMOVE_ADDR.

So mptcp_userspace_pm_delete_local_addr() helper shouldn't be called in mptcp_pm_nl_subflow_create_doit() and mptcp_pm_nl_subflow_destroy_doit().

Quoted 6 lines

Since this helper is open-coding in mptcp_pm_nl_remove_doit(), it can be dropped now.

Address entries are removed from local_addr_list when sending a REMOVE_ADDR by the userspace PM, the local_addr_used counter of PM should also be decremented accordingly.

I was re-reviewing this series after having read the comments from Sashiko:

https://sashiko.dev/#/patchset/[email protected]

I think the series is correct, but it feels like additional fixes are required:

- msk->pm.userspace_pm_local_addr_list doesn't contain an entry for the ID0 when initialised
- That's probably fine, but then it means the ID0 case needs to be
handled separately, like it is with the in-kernel PM.
- Or maybe easier to add it when the userspace PM is used, to avoid
all these exceptions?
- If yes, then the remove_doit should no longer handle the ID0
differently as well.
- same for announce_doit?
- maybe better to do that to avoid all these exceptions?

- When creating a subflow with ID0, an entry can be added to the list (to be checked)
- If the ID0 is not added at init time, then an exception should be
added there
- (also there is a check to avoid announcing ID0 → I think it is valid
to re-announce the ID0 if this address is no longer used, an exception should be added there too, no?)

- each time the list is traversed, it doesn't take into account the ID0 case:
- If the ID0 is not added at init time, then an extra check should be
added there

WDYT?

If yes, maybe this kind of patch can help? (not tested)

https://github.com/matttbe/mptcp_net-next/commit/e03ee2728e98

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
maintainer
#3 lore raw

Re: [PATCH mptcp-next v6 1/3] mptcp: pm: userspace: drop delete_local_addr helper

Hi Geliang,

Quoted 50 lines · Matthieu Baerts

Hi Geliang,

Quoted 9 lines · Geliang Tang

From: Geliang Tang <[email protected]>

Address entries should not be removed from local_addr_list when a subflow is deleted by the userspace PM, should only be removed when sending a REMOVE_ADDR.

So mptcp_userspace_pm_delete_local_addr() helper shouldn't be called in mptcp_pm_nl_subflow_create_doit() and mptcp_pm_nl_subflow_destroy_doit().

Quoted 6 lines

Since this helper is open-coding in mptcp_pm_nl_remove_doit(), it can be dropped now.

Address entries are removed from local_addr_list when sending a REMOVE_ADDR by the userspace PM, the local_addr_used counter of PM should also be decremented accordingly.

I was re-reviewing this series after having read the comments from Sashiko:

https://sashiko.dev/#/patchset/[email protected]

I think the series is correct, but it feels like additional fixes are required:

- msk->pm.userspace_pm_local_addr_list doesn't contain an entry for the ID0 when initialised
- That's probably fine, but then it means the ID0 case needs to be
handled separately, like it is with the in-kernel PM.
- Or maybe easier to add it when the userspace PM is used, to avoid
all these exceptions?
- If yes, then the remove_doit should no longer handle the ID0
differently as well.
- same for announce_doit?
- maybe better to do that to avoid all these exceptions?

- When creating a subflow with ID0, an entry can be added to the list (to be checked)
- If the ID0 is not added at init time, then an exception should be
added there
- (also there is a check to avoid announcing ID0 → I think it is valid
to re-announce the ID0 if this address is no longer used, an exception should be added there too, no?)

- each time the list is traversed, it doesn't take into account the ID0 case:
- If the ID0 is not added at init time, then an extra check should be
added there

Quoted 5 lines

WDYT?

If yes, maybe this kind of patch can help? (not tested)

https://github.com/matttbe/mptcp_net-next/commit/e03ee2728e98

I just sent this: WDYT?

https://lore.kernel.org/mptcp/20260727-mptcp-pm-userspace-id0-case-v1-0-9877f02a9bae@kernel.org/

Would it be compatible with your series?

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
author reviewer
#4 lore raw

Re: [PATCH mptcp-next v6 1/3] mptcp: pm: userspace: drop delete_local_addr helper

Hi Matt,

Quoted 75 lines · Matthieu Baerts

Hi Geliang,

Quoted 72 lines · Matthieu Baerts

Hi Geliang,

Quoted 17 lines · Geliang Tang

From: Geliang Tang <[email protected]>

Address entries should not be removed from local_addr_list when a subflow
is deleted by the userspace PM, should only be removed when sending a
REMOVE_ADDR.

So mptcp_userspace_pm_delete_local_addr() helper shouldn't be called in
mptcp_pm_nl_subflow_create_doit() and mptcp_pm_nl_subflow_destroy_doit().

Since this helper is open-coding in mptcp_pm_nl_remove_doit(), it can be
dropped now.

Quoted 5 lines

Address entries are removed from local_addr_list when sending a REMOVE_ADDR
by the userspace PM, the local_addr_used counter of PM should also be
decremented accordingly.

I was re-reviewing this series after having read the comments from Sashiko:

https://sashiko.dev/#/patchset/[email protected]

I think the series is correct, but it feels like additional fixes are
required:

- msk->pm.userspace_pm_local_addr_list doesn't contain an entry for the
ID0 when initialised
  - That's probably fine, but then it means the ID0 case needs to be
handled separately, like it is with the in-kernel PM.
  - Or maybe easier to add it when the userspace PM is used, to avoid
all these exceptions?
    - If yes, then the remove_doit should no longer handle the ID0 differently as well.
    - same for announce_doit?
    - maybe better to do that to avoid all these exceptions?

- When creating a subflow with ID0, an entry can be added to the list
(to be checked)
  - If the ID0 is not added at init time, then an exception should be
added there
  - (also there is a check to avoid announcing ID0 → I think it is valid
to re-announce the ID0 if this address is no longer used, an exception
should be added there too, no?)

- each time the list is traversed, it doesn't take into account the ID0
case:
  - If the ID0 is not added at init time, then an extra check should be
added there

WDYT?

If yes, maybe this kind of patch can help? (not tested)

Quoted 7 lines
Quoted 1 line

  https://github.com/matttbe/mptcp_net-next/commit/e03ee2728e98

I just sent this: WDYT?

https://lore.kernel.org/mptcp/20260727-mptcp-pm-userspace-id0-case-v1-0-9877f02a9bae@kernel.org/

Would it be compatible with your series?

Thank you for your patch. I will rebase my patch soon and review yours.

-Geliang

Quoted 3 lines

Cheers,
Matt

author reviewer
#5 lore raw

PATCH v6 2/3 selftests: mptcp: join: explicitly remove userspace pm addresses

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Now address entries only be removed from local_addr_list when sending a REMOVE_ADDR by the userspace PM, they're no longer removed when a subflow is deleted.

To make userspace PM selftests in mptcp_join.sh pass, this patch calls explicit userspace_pm_rm_addr() to clean up the address IDs, and adjust the chk_rm_nr counter accordingly to match the total number of RM_ADDRs sent.

---
 tools/testing/selftests/net/mptcp/mptcp_join.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/testing/selftests/net/mptcp/mptcp_join.sh +3 -1
index 550a6b6117a9..6651de43fa5c 100755
@@ -4088,11 +4088,12 @@ userspace_tests()
 		userspace_pm_chk_get_addr "${ns1}" "10" "id 10 flags signal 10.0.2.1"
 		userspace_pm_chk_get_addr "${ns1}" "20" "id 20 flags signal 10.0.3.1"
 		userspace_pm_rm_sf $ns1 "::ffff:10.0.2.1" $MPTCP_LIB_EVENT_SUB_ESTABLISHED
+		userspace_pm_rm_addr $ns1 10
 		userspace_pm_chk_dump_addr "${ns1}" \
 			"id 20 flags signal 10.0.3.1" "after rm_sf 10"
 		userspace_pm_rm_addr $ns1 20
 		userspace_pm_chk_dump_addr "${ns1}" "" "after rm_addr 20"
-		chk_rm_nr 1 1 invert
+		chk_rm_nr 2 1 invert
 		chk_mptcp_info subflows 0 subflows 0
 		chk_subflows_total 1 1
 		kill_events_pids
@@ -4118,6 +4119,7 @@ userspace_tests()
 			"subflow"
 		userspace_pm_chk_get_addr "${ns2}" "20" "id 20 flags subflow 10.0.3.2"
 		userspace_pm_rm_sf $ns2 10.0.3.2 $MPTCP_LIB_EVENT_SUB_ESTABLISHED
+		userspace_pm_rm_addr $ns2 20
 		userspace_pm_chk_dump_addr "${ns2}" \
 			"" \
 			"after rm_sf 20"
Signature · 1 line
2.53.0
author reviewer
#6 lore raw

PATCH v6 3/3 selftests: mptcp: userspace: send RM_ADDR between server and client

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Now address entries only be removed from local_addr_list when sending a REMOVE_ADDR by the userspace PM, they're no longer removed when a subflow is deleted.

To make the original userspace PM selftests pass, this patch adds explicit RM_ADDR cleanups after each subflow subtest in test_subflows(): removes the server-side local id 23 and the corresponding client-side address IDs, and also cleans up id 23 from the client side in test_subflows_v4_v6_mix(), ensuring that address entries are properly cleared to avoid interference between tests.

---
 tools/testing/selftests/net/mptcp/userspace_pm.sh | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
tools/testing/selftests/net/mptcp/userspace_pm.sh +14 -0
index e9ae1806ab07..cc6c2f3598b7 100755
@@ -603,6 +603,8 @@ test_subflows()
 	verify_subflow_events "$server_evts" "$SUB_CLOSED" "$server4_token" "$AF_INET" "10.0.2.1"\
 			      "10.0.2.2" "$client4_port" "23" "$client_addr_id" "ns1" "ns2"
 
+	# RM_ADDR from server to client machine
+	ip netns exec "$ns1" ./pm_nl_ctl rem id 23 token "$server4_token"
 	# RM_ADDR from client to server machine
 	ip netns exec "$ns2" ./pm_nl_ctl rem id $client_addr_id token\
 	   "$client4_token"
@@ -642,6 +644,8 @@ test_subflows()
 			      "dead:beef:2::1" "dead:beef:2::2" "$client6_port" "23"\
 			      "$client_addr_id" "ns1" "ns2"
 
+	# RM_ADDR from server to client machine
+	ip netns exec "$ns1" ./pm_nl_ctl rem id 23 token "$server6_token"
 	# RM_ADDR from client to server machine
 	ip netns exec "$ns2" ./pm_nl_ctl rem id $client_addr_id token\
 	   "$client6_token"
@@ -680,6 +684,8 @@ test_subflows()
 	verify_subflow_events "$server_evts" "$SUB_CLOSED" "$server4_token" "$AF_INET" "10.0.2.1"\
 			      "10.0.2.2" "$new4_port" "23" "$client_addr_id" "ns1" "ns2"
 
+	# RM_ADDR from server to client machine
+	ip netns exec "$ns1" ./pm_nl_ctl rem id 23 token "$server4_token"
 	# RM_ADDR from client to server machine
 	ip netns exec "$ns2" ./pm_nl_ctl rem id $client_addr_id token\
 	   "$client4_token"
@@ -721,6 +727,8 @@ test_subflows()
 	# RM_ADDR from server to client machine
 	ip netns exec "$ns1" ./pm_nl_ctl rem id $server_addr_id token\
 	   "$server4_token"
+	# RM_ADDR from client to server machine
+	ip netns exec "$ns2" ./pm_nl_ctl rem id 23 token "$client4_token"
 	sleep 0.5
 
 	# Attempt to add a listener at dead:beef:2::1:<subflow-port>
@@ -760,6 +768,8 @@ test_subflows()
 	# RM_ADDR6 from server to client machine
 	ip netns exec "$ns1" ./pm_nl_ctl rem id $server_addr_id token\
 	   "$server6_token"
+	# RM_ADDR6 from client to server machine
+	ip netns exec "$ns2" ./pm_nl_ctl rem id 23 token "$client6_token"
 	sleep 0.5
 
 	# Attempt to add a listener at 10.0.2.1:<new-port>
@@ -797,6 +807,8 @@ test_subflows()
 	# RM_ADDR from server to client machine
 	ip netns exec "$ns1" ./pm_nl_ctl rem id $server_addr_id token\
 	   "$server4_token"
+	# RM_ADDR from client to server machine
+	ip netns exec "$ns2" ./pm_nl_ctl rem id 23 token "$client4_token"
 }
 
 test_subflows_v4_v6_mix()
@@ -844,6 +856,8 @@ test_subflows_v4_v6_mix()
 	# RM_ADDR from server to client machine
 	ip netns exec "$ns1" ./pm_nl_ctl rem id $server_addr_id token\
 	   "$server6_token"
+	# RM_ADDR from client to server machine
+	ip netns exec "$ns2" ./pm_nl_ctl rem id 23 token "$client6_token"
 	sleep 0.5
 }
 
Signature · 1 line
2.53.0
#7 lore raw
In reply to Geliang Tang

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Unstable: 1 failed test(s): selftest_simult_flows ⚠️
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/28270421412

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/0e0044b5cd3c
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1117313

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)