Fix socket options used by NVMe over MPTCP
Subsystems (5)
Geliang Tang
reviewerv5 cover Fix socket options used by NVMe over MPTCP
From: Geliang Tang <[email protected]>
NVMe over MPTCP relies on SO_LINGER, SO_PRIORITY, SO_REUSEADDR, TCP_SYNCNT, TCP_NODELAY, IP_TOS, and SO_BINDTODEVICE. This series contains fixes to make all of them work correctly, and adds tclass support for it.
v5:
- Include Gang Yan's TCP_MAXSEG cleanup.
- Include two v4-mapped addr fixes.
- Patch 9, set rcv_flowinfo to 0 for v4-mapped addr.
- Patch 11, use rcv_flowinfo instead of np->tclass.
- All comments from Sashiko on v4 regarding "CONFIG_IPV6=m" are false positives.
v4:
- patch 1: skip ssk->sk_bound_dev_if = local->ifindex when local->ifindex
is 0, so __mptcp_subflow_connect() doesn't overwrite the inherited SO_BINDTODEVICE binding.
- patch 4: assign tcp_sock_set_syncnt()'s return to ret so an invalid TCP_SYNCNT is propagated, not silently dropped.
- patch 6: skip IPV6_TCLASS setsockopt/getsockopt paths on AF_INET sockets/subflows (inet6_sk is NULL there).
- patch 7: treat IPv4-mapped IPv6 as AF_INET for the TOS/tclass path.
- patch 9: gate ip6_sock_set_tclass() behind sk_family == AF_INET6 and IS_ENABLED(CONFIG_IPV6) to avoid IPv4 NULL-deref and link failure when CONFIG_IPV6 is off.
- patch 10: gate tclass on sk_family, skip IPv6 branch for IPv4-mapped connections, wrap IPv6 in CONFIG_IPV6.
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/
v3:
- include tclass patches.
- I also included three NVMe patches here because they have dependencies.
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/
v2:
- Drop "mptcp: don't reset dst when setting default 0 tos"
and "selftests: mptcp: sockopt: cover LINGER, REUSEADDR,
PRIORITY, NODELAY, SYNCNT": the 'if (val > 0)' guard
blocked the legitimate "reset to 0" path, and the test
only ran val_in=1, missing the SK_CAN_REUSE "any non-zero
-> 1" normalization.
- mptcp: bump setsockopt_seq for subflow-only socket options,
so secondary subflows created via MP_JOIN re-sync
sk_reuse / sk_reuseport / sk_bound_dev_if from msk.
- mptcp: copy the subflow's TOS to the msk on accept
(alongside the existing ssk->rcv_tos copy), so MP_JOIN'd
subflows inherit the reflected outgoing TOS.
- mptcp: propagate sk_reuseport to subflows via
sync_socket_options, so secondary subflows inherit
SO_REUSEPORT, not just SO_REUSEADDR.
- mptcp: tighten TCP_SYNCNT bounds check
('val < 1 || val > MAX_TCP_SYNCNT'), so out-of-bounds
values are rejected even when msk has no subflows yet
(where __mptcp_setsockopt_set_val would otherwise return 0
without invoking the set_val callback).
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/
v1:
- https://patchwork.kernel.org/project/mptcp/cover/[email protected]/
David 'equinox' Lamparter (1):
mptcp: sockopt: implement IPV6_TCLASS
Gang Yan (1):
mptcp: take TCP_MAXSEG handling into setsockopt_set_val
Geliang Tang (9):
mptcp: handle TCP_MAXSEG getsockopt in common case
mptcp: inherit sk_reuse/sk_reuseport and fix dev bind override
mptcp: add TCP_SYNCNT setsockopt/getsockopt
mptcp: skip ip6 options on v4-mapped clone
mptcp: report v4-mapped subflow addrs as IPv4
ipv6: extract and export ip6_sock_set_tclass helpers
mptcp: copy the subflow's tos/tclass to the msk on accept
nvme-tcp: support IPv6 traffic class
nvmet-tcp: support IPv6 traffic class
drivers/nvme/host/fabrics.c | 18 ++++++++ drivers/nvme/host/fabrics.h | 3 ++ drivers/nvme/host/tcp.c | 8 +++- drivers/nvme/target/tcp.c | 8 ++++ include/net/ipv6.h | 3 ++ net/ipv6/ipv6_sockglue.c | 31 +++++++++---- net/mptcp/protocol.c | 20 ++++++++- net/mptcp/sockopt.c | 89 ++++++++++++++++++++++++++++++++++--- net/mptcp/subflow.c | 4 +- 9 files changed, 166 insertions(+), 18 deletions(-)
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 1/11 mptcp: handle TCP_MAXSEG getsockopt in common case
From: Geliang Tang <[email protected]>
Move TCP_MAXSEG from a tail-clause into the common case alongside other "first subflow only" options (TCP_FASTOPEN_*), making the dispatch logic more uniform.
No behavioural change.
--- net/mptcp/sockopt.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
net/mptcp/sockopt.c +1 -3
@@ -1499,6 +1499,7 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,case TCP_FASTOPEN_CONNECT :case TCP_FASTOPEN_KEY :case TCP_FASTOPEN_NO_COOKIE : +case TCP_MAXSEG :return mptcp_getsockopt_first_sf_only (msk,SOL_TCP , optname, optval, optlen);case TCP_INQ :
@@ -1523,9 +1524,6 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,return mptcp_put_int_option (msk, optval, optlen, msk->notsent_lowat);case TCP_IS_MPTCP :return mptcp_put_int_option (msk, optval, optlen,1 ); -case TCP_MAXSEG : -return mptcp_getsockopt_first_sf_only (msk,SOL_TCP , optname, - optval, optlen); }return -EOPNOTSUPP ; }
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 2/11 mptcp: take TCP_MAXSEG handling into setsockopt_set_val
In reply to Geliang TangFrom: Gang Yan <[email protected]>
Replace mptcp_setsockopt_all_sf() with __mptcp_setsockopt_set_val() for TCP_MAXSEG option handling. This aligns the implementation with other socket options that use the same helper and simplifies the code.
--- net/mptcp/sockopt.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
net/mptcp/sockopt.c +3 -3
@@ -968,9 +968,9 @@ static int mptcp_setsockopt_sol_tcp(struct mptcp_sock *msk, int optname, val);break ;case TCP_MAXSEG : - msk->maxseg = val; - ret =mptcp_setsockopt_all_sf (msk,SOL_TCP , optname, optval, - optlen); + ret =__mptcp_setsockopt_set_val (msk,MAX_TCP_WINDOW , + &tcp_sock_set_maxseg, + &msk->maxseg, val);break ;default : ret = -ENOPROTOOPT ;
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 3/11 mptcp: inherit sk_reuse/sk_reuseport and fix dev bind override
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
Propagate sk_reuse and sk_reuseport from master socket to subflows, and bump setsockopt_seq so later subflows pick up the changes.
This patch also fixes unconditional sk_bound_dev_if override in __mptcp_subflow_connect(): only set it when local->ifindex is non-zero, otherwise preserve the value inherited from the master socket via sync_socket_options().
--- net/mptcp/sockopt.c | 5 +++++ net/mptcp/subflow.c | 4 +++- 2 files changed, 8 insertions(+), 1 deletion(-)
net/mptcp/sockopt.c +5 -0
@@ -335,6 +335,8 @@ static int mptcp_setsockopt_sol_socket(struct mptcp_sock *msk, int optname, sk->sk_bound_dev_if = ssk->sk_bound_dev_if; else if (optname == SO_BINDTOIFINDEX) sk->sk_bound_dev_if = ssk->sk_bound_dev_if; + +sockopt_seq_inc (msk); }release_sock (sk);return ret;
@@ -1704,6 +1706,9 @@ static void sync_socket_options(struct mptcp_sock *msk, struct sock *ssk)assign_bit (b, &inet_sk (ssk)->inet_flags, src &BIT (b));WRITE_ONCE (inet_sk (ssk)->local_port_range,READ_ONCE (inet_sk (sk)->local_port_range)); + + ssk->sk_reuse = sk->sk_reuse; + ssk->sk_reuseport = sk->sk_reuseport; }void mptcp_sockopt_sync_locked (struct mptcp_sock *msk,struct sock *ssk)
net/mptcp/subflow.c +3 -1
@@ -1672,7 +1672,9 @@ int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_pm_local *local,if (addr.ss_family ==AF_INET6 ) addrlen =sizeof (struct sockaddr_in6 );#endif - ssk->sk_bound_dev_if = local->ifindex; +/* Only override the bound device if the path manager picked one. */ +if (local->ifindex) + ssk->sk_bound_dev_if = local->ifindex; err =kernel_bind (sf, (struct sockaddr_unsized *)&addr, addrlen);if (err) {MPTCP_INC_STATS (sock_net (sk),MPTCP_MIB_JOINSYNTXBINDERR );
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 4/11 mptcp: add TCP_SYNCNT setsockopt/getsockopt
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
This patch adds TCP_SYNCNT support on the MPTCP socket.
The setsockopt() path stores the value in inet_csk(sk)->icsk_syn_retries on the msk and all subflow sockets. The getsockopt() path reads the same field back from the master socket.
A newly created subflow picks up the master socket's syn_retries through sync_socket_options(), so the setting takes effect for outgoing SYNs without further plumbing.
--- net/mptcp/sockopt.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
net/mptcp/sockopt.c +15 -0
@@ -974,6 +974,13 @@ static int mptcp_setsockopt_sol_tcp(struct mptcp_sock *msk, int optname, &tcp_sock_set_maxseg, &msk->maxseg, val);break ; +case TCP_SYNCNT : + ret =__mptcp_setsockopt_set_val (msk,MAX_TCP_SYNCNT , + &tcp_sock_set_syncnt, + &val, val); +if (ret ==0 ) + ret =tcp_sock_set_syncnt (sk, val); +break ;default : ret = -ENOPROTOOPT ; }
@@ -1526,6 +1533,10 @@ static int mptcp_getsockopt_sol_tcp(struct mptcp_sock *msk, int optname,return mptcp_put_int_option (msk, optval, optlen, msk->notsent_lowat);case TCP_IS_MPTCP :return mptcp_put_int_option (msk, optval, optlen,1 ); +case TCP_SYNCNT : +return mptcp_put_int_option (msk, optval, optlen, +READ_ONCE (inet_csk (sk)->icsk_syn_retries) ? : +READ_ONCE (sock_net (sk)->ipv4.sysctl_tcp_syn_retries)); }return -EOPNOTSUPP ; }
@@ -1646,6 +1657,7 @@ static void sync_socket_options(struct mptcp_sock *msk, struct sock *ssk)struct sock *sk = (struct sock *)msk;unsigned long src;bool keep_open; +int syncnt;int b; keep_open =sock_flag (sk,SOCK_KEEPOPEN );
@@ -1709,6 +1721,9 @@ static void sync_socket_options(struct mptcp_sock *msk, struct sock *ssk) ssk->sk_reuse = sk->sk_reuse; ssk->sk_reuseport = sk->sk_reuseport; + syncnt =READ_ONCE (inet_csk (sk)->icsk_syn_retries); +if (syncnt >0 &&tcp_sock_set_syncnt (ssk, syncnt)) +pr_warn ("Failed to sync TCP_SYNCNT=%u to subflow\n" , syncnt); }void mptcp_sockopt_sync_locked (struct mptcp_sock *msk,struct sock *ssk)
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 5/11 mptcp: skip ip6 options on v4-mapped clone
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
v4-mapped IPv6 destinations carry no IPv6-specific options, so skip mptcp_copy_ip6_options() when sk_v6_daddr is in ::ffff:0:0/96. Native IPv6 subflows are unchanged.
--- net/mptcp/protocol.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +2 -1
@@ -3750,7 +3750,8 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk,__mptcp_init_sock (nsk);#if IS_ENABLED (CONFIG_MPTCP_IPV6 ) -if (nsk->sk_family ==AF_INET6 ) +if (nsk->sk_family ==AF_INET6 && + !ipv6_addr_v4mapped (&nsk->sk_v6_daddr))mptcp_copy_ip6_options (nsk, sk); else#endif
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 6/11 mptcp: report v4-mapped subflow addrs as IPv4
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
mptcp_get_sub_addrs() currently fills sin6_* for any AF_INET6 subflow. Detect v4-mapped destinations (guarded by CONFIG_IPV6) and reuse the IPv4 branch so MPTCP_SUBFLOW_ADDRS reports match userspace's view.
--- net/mptcp/sockopt.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
net/mptcp/sockopt.c +6 -1
@@ -1256,10 +1256,15 @@ static int mptcp_getsockopt_tcpinfo(struct mptcp_sock *msk, char __user *optval,static void mptcp_get_sub_addrs (const struct sock *sk,struct mptcp_subflow_addrs *a) {const struct inet_sock *inet =inet_sk (sk); +bool is_v4 = sk->sk_family ==AF_INET ;memset (a,0 ,sizeof (*a)); -if (sk->sk_family ==AF_INET ) { +#if IS_ENABLED (CONFIG_IPV6 ) +if (!is_v4 && sk->sk_family ==AF_INET6 ) + is_v4 =ipv6_addr_v4mapped (&sk->sk_v6_daddr); +#endif +if (is_v4) { a->sin_local.sin_family =AF_INET ; a->sin_local.sin_port = inet->inet_sport; a->sin_local.sin_addr.s_addr = inet->inet_rcv_saddr;
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 7/11 ipv6: extract and export ip6_sock_set_tclass helpers
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
The IPV6_TCLASS case in do_ipv6_setsockopt() inlined the same ECN mask handling and inet6_sk(sk)->tclass write that callers need to do when propagating a tclass value onto a freshly created socket.
Pull this into small helpers exported via <net/ipv6.h>, so external modules (e.g. nvme-tcp, nvmet-tcp) can apply IPV6_TCLASS without duplicating the ECN handling.
--- include/net/ipv6.h | 3 +++ net/ipv6/ipv6_sockglue.c | 31 +++++++++++++++++++++++-------- 2 files changed, 26 insertions(+), 8 deletions(-)
include/net/ipv6.h +3 -0
@@ -1252,6 +1252,9 @@ static inline void ip6_sock_set_recverr(struct sock *sk)inet6_set_bit (RECVERR6 , sk); } +void __ip6_sock_set_tclass (struct sock *sk,int val); +void ip6_sock_set_tclass (struct sock *sk,int val); +#define IPV6_PREFER_SRC_MASK (IPV6_PREFER_SRC_TMP | IPV6_PREFER_SRC_PUBLIC | \ IPV6_PREFER_SRC_COA)
net/ipv6/ipv6_sockglue.c +23 -8
@@ -373,6 +373,28 @@ static int ipv6_set_opt_hdr(struct sock *sk, int optname, sockptr_t optval,return err; } +void __ip6_sock_set_tclass (struct sock *sk,int val) +{ +u8 old_tclass =inet6_sk (sk)->tclass; + +if (sk->sk_type ==SOCK_STREAM ) { + val &= ~INET_ECN_MASK ; + val |= old_tclass &INET_ECN_MASK ; + } +if (old_tclass != val) { +WRITE_ONCE (inet6_sk (sk)->tclass, val); +sk_dst_reset (sk); + } +} + +void ip6_sock_set_tclass (struct sock *sk,int val) +{ +sockopt_lock_sock (sk); +__ip6_sock_set_tclass (sk, val); +sockopt_release_sock (sk); +} +EXPORT_SYMBOL (ip6_sock_set_tclass); +int do_ipv6_setsockopt (struct sock *sk,int level,int optname,sockptr_t optval,unsigned int optlen) {
@@ -713,14 +735,7 @@ int do_ipv6_setsockopt(struct sock *sk, int level, int optname,/* RFC 3542, 6.5: default traffic class of 0x0 */ if (val ==-1 ) val =0 ; -if (sk->sk_type ==SOCK_STREAM ) { - val &= ~INET_ECN_MASK ; - val |= np->tclass &INET_ECN_MASK ; - } -if (np->tclass != val) { - np->tclass = val; -sk_dst_reset (sk); - } +__ip6_sock_set_tclass (sk, val); retv =0 ;break ;
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 8/11 mptcp: sockopt: implement IPV6_TCLASS
In reply to Geliang TangFrom: David 'equinox' Lamparter <[email protected]>
The IPV6_TCLASS setsockopt just needs to be forwarded to the individual TCP sockets, like IP_TOS is already handled for IPv4.
Coincidentally, ssh uses this sockopt and prints an error in the middle of your ongoing SSH session when it doesn't work (very annoying when doing SCP/SFTP on a multiplexed session.)
For setsockopt, the value is first applied to the MPTCP socket itself via ipv6_setsockopt(), then propagated to all existing subflows using __ip6_sock_set_tclass(). The setsockopt_seq is bumped to ensure that subflows created later will pick up the setting through sync_socket_options().
For getsockopt, the value is read directly from the MPTCP socket.
Also add sync_socket_options() support for IPV6_TCLASS, so that new subflows created after the option is set inherit the correct tclass value.
--- net/mptcp/sockopt.c | 52 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)
net/mptcp/sockopt.c +52 -0
@@ -485,6 +485,42 @@ static int mptcp_setsockopt_recverr(struct mptcp_sock *msk, int level,return ret; } +#if IS_ENABLED (CONFIG_IPV6 ) +static int mptcp_setsockopt_v6_set_tclass (struct mptcp_sock *msk,int optname, +sockptr_t optval,unsigned int optlen) +{ +struct mptcp_subflow_context *subflow; +struct sock *sk = (struct sock *)msk; +int err, val; + +if (sk->sk_family !=AF_INET6 ) +return -EOPNOTSUPP ; + + err =ipv6_setsockopt (sk,SOL_IPV6 , optname, optval, optlen); + +if (err !=0 ) +return err; + +lock_sock (sk); +sockopt_seq_inc (msk); + val =READ_ONCE (inet6_sk (sk)->tclass); +mptcp_for_each_subflow (msk, subflow) { +struct sock *ssk =mptcp_subflow_tcp_sock (subflow); +bool slow; + +if (ssk->sk_family !=AF_INET6 ) +continue ; + + slow =lock_sock_fast (ssk); +__ip6_sock_set_tclass (ssk, val); +unlock_sock_fast (ssk, slow); + } +release_sock (sk); + +return 0 ; +} +#endif +static int mptcp_setsockopt_v6 (struct mptcp_sock *msk,int optname,sockptr_t optval,unsigned int optlen) {
@@ -532,6 +568,11 @@ static int mptcp_setsockopt_v6(struct mptcp_sock *msk, int optname, ret =mptcp_setsockopt_recverr (msk,SOL_IPV6 , optname, optval, optlen);break ; +#if IS_ENABLED (CONFIG_IPV6 ) +case IPV6_TCLASS : +return mptcp_setsockopt_v6_set_tclass (msk, optname, optval, + optlen); +#endif }return ret;
@@ -1602,6 +1643,13 @@ static int mptcp_getsockopt_v6(struct mptcp_sock *msk, int optname,return -ENOPROTOOPT ;return mptcp_put_int_option (msk, optval, optlen,inet6_test_bit (RECVERR6_RFC4884 , sk)); +#if IS_ENABLED (CONFIG_IPV6 ) +case IPV6_TCLASS : +if (sk->sk_family !=AF_INET6 ) +return -EOPNOTSUPP ; +return mptcp_put_int_option (msk, optval, optlen, +READ_ONCE (inet6_sk (sk)->tclass)); +#endif }return -EOPNOTSUPP ;
@@ -1729,6 +1777,10 @@ static void sync_socket_options(struct mptcp_sock *msk, struct sock *ssk) syncnt =READ_ONCE (inet_csk (sk)->icsk_syn_retries);if (syncnt >0 &&tcp_sock_set_syncnt (ssk, syncnt))pr_warn ("Failed to sync TCP_SYNCNT=%u to subflow\n" , syncnt); +#if IS_ENABLED (CONFIG_IPV6 ) +if (sk->sk_family ==AF_INET6 && ssk->sk_family ==AF_INET6 ) +__ip6_sock_set_tclass (ssk,READ_ONCE (inet6_sk (sk)->tclass)); +#endif }void mptcp_sockopt_sync_locked (struct mptcp_sock *msk,struct sock *ssk)
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 9/11 mptcp: copy the subflow's tos/tclass to the msk on accept
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
When a new MPTCP socket is accepted, copy the tos (and rcv_tos for IPv4) or tclass (and rcv_flowinfo for IPv6) from the first subflow to the newly-created msk.
This mirrors what plain TCP does in tcp_v4_syn_recv_sock() and tcp_v6_syn_recv_sock(), where these values are copied from the incoming skb to the accepted socket.
Without this, MP_JOIN subflows created later would inherit tos/tclass=0 from the msk and lose the peer's reflected value, causing inconsistent traffic class behavior.
--- net/mptcp/protocol.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
net/mptcp/protocol.c +17 -0
@@ -3757,6 +3757,23 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk,#endif mptcp_copy_ip_options (nsk, sk); +if (ssk->sk_family ==AF_INET ) { +inet_sk (nsk)->rcv_tos =inet_sk (ssk)->rcv_tos; +__ip_sock_set_tos (nsk,inet_sk (ssk)->tos); + } +#if IS_ENABLED(CONFIG_MPTCP_IPV6) + else if (ssk->sk_family == AF_INET6 && + ipv6_addr_v4mapped(&ssk->sk_v6_daddr)) { +inet_sk (nsk)->rcv_tos =inet_sk (ssk)->rcv_tos; +__ip_sock_set_tos (nsk,inet_sk (ssk)->tos); +inet6_sk (nsk)->rcv_flowinfo =0 ; + } + else if (ssk->sk_family == AF_INET6) { +inet6_sk (nsk)->rcv_flowinfo =inet6_sk (ssk)->rcv_flowinfo; +__ip6_sock_set_tclass (nsk,inet6_sk (ssk)->tclass); + } +#endif + msk =mptcp_sk (nsk);WRITE_ONCE (msk->local_key, subflow_req->local_key);WRITE_ONCE (msk->token, subflow_req->token);
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 10/11 nvme-tcp: support IPv6 traffic class
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
The NVMe/TCP transport needs to configure the IPv6 traffic class on its queue sockets, but the fabrics option parser currently has no corresponding option.
Add the tclass option to the fabrics parser and store the validated value in struct nvmf_ctrl_options. Negative values are rejected, while values greater than 255 are clamped to 255, matching the valid range of the IPv6 traffic class field.
Keep -1 as the default value to let the transport use its default traffic class.
The fabrics layer now supports parsing the tclass option, but the NVMe/TCP transport does not currently advertise or apply it.
Allow NVMe/TCP to use NVMF_OPT_TCLASS and set IPV6_TCLASS on each queue socket when a traffic class is specified. Keep the existing IPv4 TOS handling unchanged.
--- drivers/nvme/host/fabrics.c | 18 ++++++++++++++++++ drivers/nvme/host/fabrics.h | 3 +++ drivers/nvme/host/tcp.c | 8 +++++++- 3 files changed, 28 insertions(+), 1 deletion(-)
drivers/nvme/host/fabrics.c +18 -0
@@ -695,6 +695,7 @@ static const match_table_t opt_tokens = { { NVMF_OPT_NR_WRITE_QUEUES, "nr_write_queues=%d" }, { NVMF_OPT_NR_POLL_QUEUES, "nr_poll_queues=%d" }, { NVMF_OPT_TOS, "tos=%d" }, + { NVMF_OPT_TCLASS, "tclass=%d" }, #ifdef CONFIG_NVME_TCP_TLS { NVMF_OPT_KEYRING, "keyring=%d" }, { NVMF_OPT_TLS_KEY, "tls_key=%d" },
@@ -734,6 +735,7 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts, opts->hdr_digest = false; opts->data_digest = false; opts->tos =-1 ;/* < 0 == use transport default */ + opts->tclass =-1 ;/* < 0 == use transport default */ opts->tls = false; opts->tls_key =NULL ; opts->keyring =NULL ;
@@ -991,6 +993,22 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts, } opts->tos = token;break ; +case NVMF_OPT_TCLASS : +if (match_int (args, &token)) { + ret = -EINVAL ; + goto out; + } +if (token <0 ) { +pr_err ("Invalid traffic class %d\n" , token); + ret = -EINVAL ; + goto out; + } +if (token >255 ) { +pr_warn ("Clamping traffic class to 255\n" ); + token =255 ; + } + opts->tclass = token; +break ;case NVMF_OPT_KEYRING :if (match_int (args, &key_id) || key_id <=0 ) { ret = -EINVAL ;
drivers/nvme/host/fabrics.h +3 -0
@@ -67,6 +67,7 @@ enum {NVMF_OPT_KEYRING =1 <<26 ,NVMF_OPT_TLS_KEY =1 <<27 ,NVMF_OPT_CONCAT =1 <<28 , + NVMF_OPT_TCLASS = 1 << 29, };/**
@@ -109,6 +110,7 @@ enum {* @nr_write_queues: number of queues for write I/O * @nr_poll_queues: number of queues for polling I/O * @tos: type of service +* @tclass: IPv6 traffic class * @fast_io_fail_tmo: Fast I/O fail timeout in seconds */ struct nvmf_ctrl_options {
@@ -139,6 +141,7 @@ struct nvmf_ctrl_options {unsigned int nr_write_queues;unsigned int nr_poll_queues;int tos; +int tclass;int fast_io_fail_tmo; };
drivers/nvme/host/tcp.c +7 -1
@@ -1837,6 +1837,11 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,/* Set socket type of service */ if (nctrl->opts->tos >=0 )ip_sock_set_tos (queue->sock->sk, nctrl->opts->tos); +#if IS_ENABLED (CONFIG_IPV6 ) +if (nctrl->opts->tclass >=0 && + queue->sock->sk->sk_family ==AF_INET6 ) +ip6_sock_set_tclass (queue->sock->sk, nctrl->opts->tclass); +#endif /* Set 10 seconds timeout for icresp recvmsg */ queue->sock->sk->sk_rcvtimeo =10 *HZ ;
@@ -3041,7 +3046,8 @@ static struct nvmf_transport_ops nvme_tcp_transport = {NVMF_OPT_HOST_TRADDR |NVMF_OPT_CTRL_LOSS_TMO |NVMF_OPT_HDR_DIGEST |NVMF_OPT_DATA_DIGEST |NVMF_OPT_NR_WRITE_QUEUES |NVMF_OPT_NR_POLL_QUEUES | -NVMF_OPT_TOS |NVMF_OPT_HOST_IFACE |NVMF_OPT_TLS | +NVMF_OPT_TOS |NVMF_OPT_TCLASS | +NVMF_OPT_HOST_IFACE |NVMF_OPT_TLS | NVMF_OPT_KEYRING | NVMF_OPT_TLS_KEY | NVMF_OPT_CONCAT, .create_ctrl = nvme_tcp_create_ctrl, };
Signature · 1 line
2.53.0
Geliang Tang
reviewerPATCH v5 11/11 nvmet-tcp: support IPv6 traffic class
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
nvmet-tcp currently applies the received IPv4 TOS value when setting up a queue socket, but does not handle the IPv6 traffic class.
Extend the queue socket setup to handle AF_INET6 sockets. Obtain the traffic class from the IPv6 socket and apply it through IPV6_TCLASS. Keep the existing IPv4 TOS handling unchanged.
--- drivers/nvme/target/tcp.c | 8 ++++++++ 1 file changed, 8 insertions(+)
drivers/nvme/target/tcp.c +8 -0
@@ -1725,6 +1725,14 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)/* Set socket type of service */ if (inet->rcv_tos >0 )ip_sock_set_tos (sock->sk, inet->rcv_tos); +#if IS_ENABLED (CONFIG_IPV6 ) +if (sock->sk->sk_family ==AF_INET6 ) { +u8 rcv_tclass =ip6_tclass (inet6_sk (sock->sk)->rcv_flowinfo); + +if (rcv_tclass >0 ) +ip6_sock_set_tclass (sock->sk, rcv_tclass); + } +#endif ret =0 ;write_lock_bh (&sock->sk->sk_callback_lock);
Signature · 1 line
2.53.0
Hi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/31238427573
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/b9ff91daca52
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1142524
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-normalFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)