Skip to contentSkip to conversation

Fix socket options used by NVMe over MPTCP

PATCH Superseded Geliang Tang 2 participants 7 messages Jul 26

Subsystems (2)

v2 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Geliang Tang

author reviewer
#0 lore raw

v2 cover Fix socket options used by NVMe over MPTCP

From: Geliang Tang <[email protected]>

NVMe over MPTCP relies on SO_LINGER, SO_PRIORITY, SO_REUSEADDR, TCP_SYNCNT, TCP_NODELAY, IP_TOS, and SO_BINDTODEVICE. This series contains fixes to make all of them work correctly. Patch 4 is a cleanup needed by patch 5.

v2:
- Drop "mptcp: don't reset dst when setting default 0 tos" and "selftests: mptcp: sockopt: cover LINGER, REUSEADDR, PRIORITY, NODELAY, SYNCNT": the 'if (val > 0)' guard blocked the legitimate "reset to 0" path, and the test only ran val_in=1, missing the SK_CAN_REUSE "any non-zero -> 1" normalization.
- mptcp: bump setsockopt_seq for subflow-only socket options, so secondary subflows created via MP_JOIN re-sync sk_reuse / sk_reuseport / sk_bound_dev_if from msk.
- mptcp: copy the subflow's TOS to the msk on accept (alongside the existing ssk->rcv_tos copy), so MP_JOIN'd subflows inherit the reflected outgoing TOS.
- mptcp: propagate sk_reuseport to subflows via sync_socket_options, so secondary subflows inherit SO_REUSEPORT, not just SO_REUSEADDR.
- mptcp: tighten TCP_SYNCNT bounds check
('val < 1 || val > MAX_TCP_SYNCNT'), so out-of-bounds values are rejected even when msk has no subflows yet (where __mptcp_setsockopt_set_val would otherwise return 0 without invoking the set_val callback).

v1:
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/

Geliang Tang (5):
mptcp: copy the subflow's tos to the msk on accept
mptcp: inherit sk_reuse/sk_reuseport on subflow creation
mptcp: bump setsockopt_seq for subflow-only socket options
mptcp: handle TCP_MAXSEG getsockopt in common case
mptcp: add TCP_SYNCNT setsockopt/getsockopt

 net/mptcp/protocol.c |  3 +++
 net/mptcp/sockopt.c  | 35 ++++++++++++++++++++++++++++++++---
 2 files changed, 35 insertions(+), 3 deletions(-)
Signature · 1 line
2.53.0
author reviewer
#1 lore raw

PATCH v2 1/5 mptcp: copy the subflow's tos to the msk on accept

From: Geliang Tang <[email protected]>

mptcp_sk_clone_init() now copies both inet->tos and inet->rcv_tos from the first subflow onto the accepted msk, mirroring plain TCP's tcp_v4_syn_recv_sock() which sets both fields from ip_hdr(skb)->tos on the accepted socket. Without this, MP_JOIN'd subflows would inherit tos=0 from the msk and lose the peer's reflected value.

---
 net/mptcp/protocol.c | 3 +++
 1 file changed, 3 insertions(+)
net/mptcp/protocol.c +3 -0
index 954e20bb27de..1152f1d0c24f 100644
@@ -3699,6 +3699,9 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk,
 #endif
 		mptcp_copy_ip_options(nsk, sk);
 
+	inet_sk(nsk)->rcv_tos = inet_sk(ssk)->rcv_tos;
+	inet_sk(nsk)->tos = inet_sk(ssk)->tos;
+
 	msk = mptcp_sk(nsk);
 	WRITE_ONCE(msk->local_key, subflow_req->local_key);
 	WRITE_ONCE(msk->token, subflow_req->token);
Signature · 1 line
2.53.0
author reviewer
#2 lore raw

PATCH v2 2/5 mptcp: inherit sk_reuse/sk_reuseport on subflow creation

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Propagate sk_reuse and sk_reuseport from the MPTCP master socket to newly-created subflows in sync_socket_options(), mirroring how sk_reuse and sk_reuseport are already handled for the subflow.

Without this, a subflow started after the master toggled SO_REUSEADDR or SO_REUSEPORT would not inherit the setting, leading to inconsistent reuse semantics between msk and ssk.

---
 net/mptcp/sockopt.c | 3 +++
 1 file changed, 3 insertions(+)
net/mptcp/sockopt.c +3 -0
index fcf6feb2a9eb..dc1ef52c2200 100644
@@ -1601,6 +1601,9 @@ static void sync_socket_options(struct mptcp_sock *msk, struct sock *ssk)
 	inet_assign_bit(FREEBIND, ssk, inet_test_bit(FREEBIND, sk));
 	inet_assign_bit(BIND_ADDRESS_NO_PORT, ssk, inet_test_bit(BIND_ADDRESS_NO_PORT, sk));
 	WRITE_ONCE(inet_sk(ssk)->local_port_range, READ_ONCE(inet_sk(sk)->local_port_range));
+
+	ssk->sk_reuse = sk->sk_reuse;
+	ssk->sk_reuseport = sk->sk_reuseport;
 }
 
 void mptcp_sockopt_sync_locked(struct mptcp_sock *msk, struct sock *ssk)
Signature · 1 line
2.53.0
author reviewer
#3 lore raw

PATCH v2 3/5 mptcp: bump setsockopt_seq for subflow-only socket options

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

mptcp_setsockopt_sol_socket()'s SO_REUSEPORT/SO_REUSEADDR/SO_BINDTODEVICE /SO_BINDTOIFINDEX case applies the option to the nmpc subflow and copies the result back onto msk, then early-returns without calling sockopt_seq_inc(msk).

That leaves msk->setsockopt_seq unchanged, so any subflow created later (e.g. via MP_JOIN) inherits the master socket's seq at creation time, and mptcp_sockopt_sync_locked() sees 'seq matches' and skips sync_socket_options() entirely. As a result, the new subflow's sk_reuse /sk_reuseport/sk_bound_dev_if stay at the listener's default of 0 even though the nmpc subflow (and msk) have the right value.

Bump the seq on this path so secondary subflows trigger the sync on join. The actual field propagation lives in sync_socket_options(); this commit only flips the trigger.

---
 net/mptcp/sockopt.c | 2 ++
 1 file changed, 2 insertions(+)
net/mptcp/sockopt.c +2 -0
index dc1ef52c2200..37e65923f544 100644
@@ -326,6 +326,8 @@ static int mptcp_setsockopt_sol_socket(struct mptcp_sock *msk, int optname,
 				sk->sk_bound_dev_if = ssk->sk_bound_dev_if;
 			else if (optname == SO_BINDTOIFINDEX)
 				sk->sk_bound_dev_if = ssk->sk_bound_dev_if;
+
+			sockopt_seq_inc(msk);
 		}
 		release_sock(sk);
 		return ret;
Signature · 1 line
2.53.0
author reviewer
#4 lore raw

PATCH v2 4/5 mptcp: handle TCP_MAXSEG getsockopt in common case

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

The TCP_MAXSEG getsockopt() case was a tail-clause returning mptcp_getsockopt_first_sf_only() while every other "first subflow only" option (TCP_FASTOPEN_*) was already grouped in the common case. Move it next to its peers so the dispatch reads uniformly.

No behavioural change; the lookup still returns the cached MSS on the first subflow.

---
 net/mptcp/sockopt.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
net/mptcp/sockopt.c +1 -3
index 37e65923f544..34f6cd9c7967 100644
@@ -1425,6 +1425,7 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 	case TCP_FASTOPEN_CONNECT:
 	case TCP_FASTOPEN_KEY:
 	case TCP_FASTOPEN_NO_COOKIE:
+	case TCP_MAXSEG:
 		return mptcp_getsockopt_first_sf_only(msk, SOL_TCP, optname,
 						      optval, optlen);
 	case TCP_INQ:
@@ -1449,9 +1450,6 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 		return mptcp_put_int_option(msk, optval, optlen, msk->notsent_lowat);
 	case TCP_IS_MPTCP:
 		return mptcp_put_int_option(msk, optval, optlen, 1);
-	case TCP_MAXSEG:
-		return mptcp_getsockopt_first_sf_only(msk, SOL_TCP, optname,
-						      optval, optlen);
 	}
 	return -EOPNOTSUPP;
 }
Signature · 1 line
2.53.0
author reviewer
#5 lore raw

PATCH v2 5/5 mptcp: add TCP_SYNCNT setsockopt/getsockopt

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Surface TCP_SYNCNT to userspace on the MPTCP master socket.

setsockopt(TCP_SYNCNT) validates the value via __mptcp_setsockopt_set_val() and stores it in inet_csk(sk)->icsk_syn_retries on the msk. getsockopt() reads the same field back.

A newly created subflow picks up the master socket's syn_retries through sync_socket_options(), so the setting takes effect for outgoing SYNs without further plumbing.

---
 net/mptcp/sockopt.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
net/mptcp/sockopt.c +26 -0
index 34f6cd9c7967..6e4c5b5314af 100644
@@ -898,6 +898,23 @@ static int mptcp_setsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 		ret = mptcp_setsockopt_all_sf(msk, SOL_TCP, optname, optval,
 					      optlen);
 		break;
+	case TCP_SYNCNT: {
+		int syncnt;
+
+		if (val < 1 || val > MAX_TCP_SYNCNT) {
+			ret = -EINVAL;
+			break;
+		}
+
+		syncnt = val;
+		ret = __mptcp_setsockopt_set_val(msk, MAX_TCP_SYNCNT,
+						 &tcp_sock_set_syncnt,
+						 &syncnt,
+						 val);
+		if (ret == 0)
+			inet_csk(sk)->icsk_syn_retries = syncnt;
+		break;
+	}
 	default:
 		ret = -ENOPROTOOPT;
 	}
@@ -1450,6 +1467,10 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,
 		return mptcp_put_int_option(msk, optval, optlen, msk->notsent_lowat);
 	case TCP_IS_MPTCP:
 		return mptcp_put_int_option(msk, optval, optlen, 1);
+	case TCP_SYNCNT:
+		return mptcp_put_int_option(msk, optval, optlen,
+					    inet_csk(sk)->icsk_syn_retries ? :
+					    READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_syn_retries));
 	}
 	return -EOPNOTSUPP;
 }
@@ -1604,6 +1625,11 @@ static void sync_socket_options(struct mptcp_sock *msk, struct sock *ssk)
 
 	ssk->sk_reuse = sk->sk_reuse;
 	ssk->sk_reuseport = sk->sk_reuseport;
+	if (inet_csk(sk)->icsk_syn_retries > 0) {
+		if (tcp_sock_set_syncnt(ssk, inet_csk(sk)->icsk_syn_retries))
+			pr_warn("Failed to sync TCP_SYNCNT=%u to subflow\n",
+				inet_csk(sk)->icsk_syn_retries);
+	}
 }
 
 void mptcp_sockopt_sync_locked(struct mptcp_sock *msk, struct sock *ssk)
Signature · 1 line
2.53.0
#6 lore raw
In reply to Geliang Tang

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/30195236916

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/006edc38647a
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1134602

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)