Skip to contentSkip to conversation

nvme-tcp: add IPv6 traffic class support

PATCH Superseded Geliang Tang 4 participants 9 messages Aug 8 – Aug 11

Subsystems (6)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Geliang Tang

author reviewer
#0 lore raw

v2 cover nvme-tcp: add IPv6 traffic class support

From: Geliang Tang <[email protected]>

This series adds IPv6 traffic class (tclass) support to the NVMe/TCP fabrics stack, mirroring the existing IPv4 TOS configuration.

The current code path only handles IPv4: nvme-fabrics exposes a 'tos' option, the host (nvme-tcp) applies IP_TOS to each queue socket, and the target (nvmet-tcp) reflects the value received on the wire. This leaves no way to control or observe the IPv6 traffic class for NVMe/TCP connections, which is a real gap for deployments running over IPv6 or MPTCP subflows that need a non-default tclass.

The series is split into three patches following the natural fabrics/host/target layering:

1. nvme-fabrics: add IPv6 traffic class option
Parse the new 'tclass' option in nvmf_parse_options(), validate that it is in the 0-255 range, and store it in struct nvmf_ctrl_options. Use -1 to mean "transport default", matching the existing tos handling.

2. nvme-tcp: support IPv6 traffic class
Advertise NVMF_OPT_TCLASS in the nvme_tcp transport and, when a tclass was provided, set IPV6_TCLASS on each queue socket. The existing IP_TOS path is left untouched so IPv4 behaviour is unchanged.

3. nvmet-tcp: support IPv6 traffic class
On the target side, differentiate the socket family: for AF_INET keep the existing TOS handling, and for AF_INET6 read the traffic class from the incoming socket and apply it via IPV6_TCLASS.

Follow-up changes outside of this series expose the same option through libnvme and the nvme-cli, so the new tclass can be selected from the user-facing command line; those patches will be sent in a separate series.

This was tested with the selftests mptcp_nvme.sh [1], extended in "nvme: add tclass support" to pass "--tclass" on the host side and validate that the value is reflected on the wire by the target.

v2:
- Export and use the ip6_sock_set_tclass() helper in both target and host paths, aligning with the existing ip_sock_set_tos() usage.
- Add CONFIG_IPV6 and AF_INET6 guards to avoid build failures when IPv6 is disabled.
- Use rcv_flowinfo from the target socket instead of np->tclass.

v1:
- https://patchwork.kernel.org/project/linux-nvme/cover/[email protected]/

[1]
https://patchwork.kernel.org/project/linux-nvme/cover/[email protected]/

Geliang Tang (3):
ipv6: extract and export ip6_sock_set_tclass helpers
nvme-tcp: support IPv6 traffic class
nvmet-tcp: support IPv6 traffic class

 drivers/nvme/host/fabrics.c | 18 ++++++++++++++++++
 drivers/nvme/host/fabrics.h |  3 +++
 drivers/nvme/host/tcp.c     |  8 +++++++-
 drivers/nvme/target/tcp.c   |  8 ++++++++
 include/net/ipv6.h          |  3 +++
 net/ipv6/ipv6_sockglue.c    | 31 +++++++++++++++++++++++--------
 6 files changed, 62 insertions(+), 9 deletions(-)
Signature · 1 line
2.53.0
author reviewer
#1 lore raw

PATCH v2 1/3 ipv6: extract and export ip6_sock_set_tclass helpers

From: Geliang Tang <[email protected]>

IPV6_TCLASS handling in do_ipv6_setsockopt() inlines the same ECN mask logic and inet6_sk(sk)->tclass write that callers need when propagating a tclass value onto a newly created socket.

Pull this into small helpers __ip6_sock_set_tclass()/ip6_sock_set_tclass() exported via <net/ipv6.h>, so external modules can apply IPV6_TCLASS without duplicating the ECN handling.

__ip6_sock_set_tclass() will be used by MPTCP's sockopt implementation (similar to __ip_sock_set_tos), while ip6_sock_set_tclass() will be used by nvme-tcp (similar to ip_sock_set_tos).

---
 include/net/ipv6.h       |  3 +++
 net/ipv6/ipv6_sockglue.c | 31 +++++++++++++++++++++++--------
 2 files changed, 26 insertions(+), 8 deletions(-)
include/net/ipv6.h +3 -0
index 3de07e738538..82e57b34ba57 100644
@@ -1252,6 +1252,9 @@ static inline void ip6_sock_set_recverr(struct sock *sk)
 	inet6_set_bit(RECVERR6, sk);
 }
 
+void __ip6_sock_set_tclass(struct sock *sk, int val);
+void ip6_sock_set_tclass(struct sock *sk, int val);
+
 #define IPV6_PREFER_SRC_MASK (IPV6_PREFER_SRC_TMP | IPV6_PREFER_SRC_PUBLIC | \
 			      IPV6_PREFER_SRC_COA)
 
net/ipv6/ipv6_sockglue.c +23 -8
index b4c977434c2e..c59f25fbb336 100644
@@ -373,6 +373,28 @@ static int ipv6_set_opt_hdr(struct sock *sk, int optname, sockptr_t optval,
 	return err;
 }
 
+void __ip6_sock_set_tclass(struct sock *sk, int val)
+{
+	u8 old_tclass = inet6_sk(sk)->tclass;
+
+	if (sk->sk_type == SOCK_STREAM) {
+		val &= ~INET_ECN_MASK;
+		val |= old_tclass & INET_ECN_MASK;
+	}
+	if (old_tclass != val) {
+		WRITE_ONCE(inet6_sk(sk)->tclass, val);
+		sk_dst_reset(sk);
+	}
+}
+
+void ip6_sock_set_tclass(struct sock *sk, int val)
+{
+	sockopt_lock_sock(sk);
+	__ip6_sock_set_tclass(sk, val);
+	sockopt_release_sock(sk);
+}
+EXPORT_SYMBOL(ip6_sock_set_tclass);
+
 int do_ipv6_setsockopt(struct sock *sk, int level, int optname,
 		       sockptr_t optval, unsigned int optlen)
 {
@@ -713,14 +735,7 @@ int do_ipv6_setsockopt(struct sock *sk, int level, int optname,
 		/* RFC 3542, 6.5: default traffic class of 0x0 */
 		if (val == -1)
 			val = 0;
-		if (sk->sk_type == SOCK_STREAM) {
-			val &= ~INET_ECN_MASK;
-			val |= np->tclass & INET_ECN_MASK;
-		}
-		if (np->tclass != val) {
-			np->tclass = val;
-			sk_dst_reset(sk);
-		}
+		__ip6_sock_set_tclass(sk, val);
 		retv = 0;
 		break;
 
Signature · 1 line
2.53.0
maintainer
#2 lore raw

Re: [PATCH v2 1/3] ipv6: extract and export ip6_sock_set_tclass helpers

Quoted 10 lines · On Sat, 8 Aug 2026 14:54:59 +0800 Geliang Tang wrote:

IPV6_TCLASS handling in do_ipv6_setsockopt() inlines the same ECN mask logic and inet6_sk(sk)->tclass write that callers need when propagating a tclass value onto a newly created socket.

Pull this into small helpers __ip6_sock_set_tclass()/ip6_sock_set_tclass() exported via <net/ipv6.h>, so external modules can apply IPV6_TCLASS without duplicating the ECN handling.

__ip6_sock_set_tclass() will be used by MPTCP's sockopt implementation (similar to __ip_sock_set_tos),

not part of this series

Quoted 2 lines

while ip6_sock_set_tclass() will be used
by nvme-tcp (similar to ip_sock_set_tos).

setsockopt already supports generic kernel callers, sth like:

do_sock_setsockopt(sock, false, SOL_IPV6, IPV6_TCLASS,
KERNEL_SOCKPTR(&val), sizeof(val));

? Please don't export helpers for every single sockopt.

author reviewer
#3 lore raw

Re: [PATCH v2 1/3] ipv6: extract and export ip6_sock_set_tclass helpers

Hi Jakub,

Quoted 13 lines · Jakub Kicinski
Quoted 12 lines · On Sat,  8 Aug 2026 14:54:59 +0800 Geliang Tang wrote:

IPV6_TCLASS handling in do_ipv6_setsockopt() inlines the same ECN mask
logic and inet6_sk(sk)->tclass write that callers need when propagating
a tclass value onto a newly created socket.

Pull this into small helpers __ip6_sock_set_tclass()/ip6_sock_set_tclass() exported via <net/ipv6.h>, so external modules can apply IPV6_TCLASS
without duplicating the ECN handling.

Quoted 5 lines
Quoted 3 lines

__ip6_sock_set_tclass() will be used by MPTCP's sockopt implementation
(similar to __ip_sock_set_tos),

not part of this series

Exactly. The export of __ip6_sock_set_tclass() is not part of this series, it belongs to the MPTCP series that adds IPV6_TCLASS support. I will move this export to that series instead.

Quoted 10 lines
Quoted 3 lines

while ip6_sock_set_tclass() will be used
by nvme-tcp (similar to ip_sock_set_tos).

setsockopt already supports generic kernel callers, sth like:

   do_sock_setsockopt(sock, false, SOL_IPV6, IPV6_TCLASS, KERNEL_SOCKPTR(&val), sizeof(val));

? Please don't export helpers for every single sockopt.

That is exactly what I planned to do in a follow-up patch - please see my reply to [1].

The goal is to make NVMe work with both TCP and MPTCP going forward. To achieve that, all sockopt calls in NVMe need to be switched to use do_sock_setsockopt() instead of calling individual sockopt helpers. This approach abstracts away the protocol differences, so NVMe no longer needs to care whether it is dealing with TCP or MPTCP.

Even when MPTCP is not involved, switching to do_sock_setsockopt() provides a general improvement to the code. I will include this "switch NVMe to do_sock_setsockopt" patch in v3 of this series.

Thanks,
-Geliang

[1]https://patchwork.kernel.org/project/linux-nvme/patch/a1fa5dd3ebb0cafe1bf7948bf2a96436e9452e19.1779934709.git.tanggeliang@kylinos.cn/

author reviewer
#4 lore raw

PATCH v2 2/3 nvme-tcp: support IPv6 traffic class

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

The NVMe/TCP transport needs to configure the IPv6 traffic class on its queue sockets, but the fabrics option parser currently has no corresponding option.

Add the tclass option to the fabrics parser and store the validated value in struct nvmf_ctrl_options. Negative values are rejected, while values greater than 255 are clamped to 255, matching the valid range of the IPv6 traffic class field.

Keep -1 as the default value to let the transport use its default traffic class.

The fabrics layer now supports parsing the tclass option, but the NVMe/TCP transport does not currently advertise or apply it.

Allow NVMe/TCP to use NVMF_OPT_TCLASS and set IPV6_TCLASS on each queue socket via ip6_sock_set_tclass() when a traffic class is specified, keeping the existing IPv4 TOS handling unchanged.

---
 drivers/nvme/host/fabrics.c | 18 ++++++++++++++++++
 drivers/nvme/host/fabrics.h |  3 +++
 drivers/nvme/host/tcp.c     |  8 +++++++-
 3 files changed, 28 insertions(+), 1 deletion(-)
drivers/nvme/host/fabrics.c +18 -0
index ac3d4f400601..643c03dc7bcb 100644
@@ -695,6 +695,7 @@ static const match_table_t opt_tokens = {
 	{ NVMF_OPT_NR_WRITE_QUEUES,	"nr_write_queues=%d"	},
 	{ NVMF_OPT_NR_POLL_QUEUES,	"nr_poll_queues=%d"	},
 	{ NVMF_OPT_TOS,			"tos=%d"		},
+	{ NVMF_OPT_TCLASS,		"tclass=%d"		},
 #ifdef CONFIG_NVME_TCP_TLS
 	{ NVMF_OPT_KEYRING,		"keyring=%d"		},
 	{ NVMF_OPT_TLS_KEY,		"tls_key=%d"		},
@@ -734,6 +735,7 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
 	opts->hdr_digest = false;
 	opts->data_digest = false;
 	opts->tos = -1; /* < 0 == use transport default */
+	opts->tclass = -1; /* < 0 == use transport default */
 	opts->tls = false;
 	opts->tls_key = NULL;
 	opts->keyring = NULL;
@@ -991,6 +993,22 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
 			}
 			opts->tos = token;
 			break;
+		case NVMF_OPT_TCLASS:
+			if (match_int(args, &token)) {
+				ret = -EINVAL;
+				goto out;
+			}
+			if (token < 0) {
+				pr_err("Invalid traffic class %d\n", token);
+				ret = -EINVAL;
+				goto out;
+			}
+			if (token > 255) {
+				pr_warn("Clamping traffic class to 255\n");
+				token = 255;
+			}
+			opts->tclass = token;
+			break;
 		case NVMF_OPT_KEYRING:
 			if (match_int(args, &key_id) || key_id <= 0) {
 				ret = -EINVAL;
drivers/nvme/host/fabrics.h +3 -0
index caf5503d0833..3dfd40bd8960 100644
@@ -67,6 +67,7 @@ enum {
 	NVMF_OPT_KEYRING	= 1 << 26,
 	NVMF_OPT_TLS_KEY	= 1 << 27,
 	NVMF_OPT_CONCAT		= 1 << 28,
+	NVMF_OPT_TCLASS		= 1 << 29,
 };
 
 /**
@@ -109,6 +110,7 @@ enum {
  * @nr_write_queues: number of queues for write I/O
  * @nr_poll_queues: number of queues for polling I/O
  * @tos: type of service
+ * @tclass: IPv6 traffic class
  * @fast_io_fail_tmo: Fast I/O fail timeout in seconds
  */
 struct nvmf_ctrl_options {
@@ -139,6 +141,7 @@ struct nvmf_ctrl_options {
 	unsigned int		nr_write_queues;
 	unsigned int		nr_poll_queues;
 	int			tos;
+	int			tclass;
 	int			fast_io_fail_tmo;
 };
 
drivers/nvme/host/tcp.c +7 -1
index ba5c7b3e2a7c..cabbf67d24f8 100644
@@ -1837,6 +1837,11 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 	/* Set socket type of service */
 	if (nctrl->opts->tos >= 0)
 		ip_sock_set_tos(queue->sock->sk, nctrl->opts->tos);
+#if IS_ENABLED(CONFIG_IPV6)
+	if (nctrl->opts->tclass >= 0 &&
+	    queue->sock->sk->sk_family == AF_INET6)
+		ip6_sock_set_tclass(queue->sock->sk, nctrl->opts->tclass);
+#endif
 
 	/* Set 10 seconds timeout for icresp recvmsg */
 	queue->sock->sk->sk_rcvtimeo = 10 * HZ;
@@ -3041,7 +3046,8 @@ static struct nvmf_transport_ops nvme_tcp_transport = {
 			  NVMF_OPT_HOST_TRADDR | NVMF_OPT_CTRL_LOSS_TMO |
 			  NVMF_OPT_HDR_DIGEST | NVMF_OPT_DATA_DIGEST |
 			  NVMF_OPT_NR_WRITE_QUEUES | NVMF_OPT_NR_POLL_QUEUES |
-			  NVMF_OPT_TOS | NVMF_OPT_HOST_IFACE | NVMF_OPT_TLS |
+			  NVMF_OPT_TOS | NVMF_OPT_TCLASS |
+			  NVMF_OPT_HOST_IFACE | NVMF_OPT_TLS |
 			  NVMF_OPT_KEYRING | NVMF_OPT_TLS_KEY | NVMF_OPT_CONCAT,
 	.create_ctrl	= nvme_tcp_create_ctrl,
 };
Signature · 1 line
2.53.0
#5 lore raw

Re: [PATCH v2 2/3] nvme-tcp: support IPv6 traffic class

Quoted 33 lines · Geliang Tang

From: Geliang Tang <[email protected]>

The NVMe/TCP transport needs to configure the IPv6 traffic class on its queue sockets, but the fabrics option parser currently has no corresponding option.

Add the tclass option to the fabrics parser and store the validated value in struct nvmf_ctrl_options. Negative values are rejected, while values greater than 255 are clamped to 255, matching the valid range of the IPv6 traffic class field.

Keep -1 as the default value to let the transport use its default traffic class.

The fabrics layer now supports parsing the tclass option, but the NVMe/TCP transport does not currently advertise or apply it.

Allow NVMe/TCP to use NVMF_OPT_TCLASS and set IPV6_TCLASS on each queue socket via ip6_sock_set_tclass() when a traffic class is specified, keeping the existing IPv4 TOS handling unchanged.

---
 drivers/nvme/host/fabrics.c | 18 ++++++++++++++++++
 drivers/nvme/host/fabrics.h |  3 +++
 drivers/nvme/host/tcp.c     |  8 +++++++-
 3 files changed, 28 insertions(+), 1 deletion(-)
drivers/nvme/host/fabrics.c +0 -0
index ac3d4f400601..643c03dc7bcb 100644

drivers/nvme/host/fabrics.c static const match_table_t opt_tokens = { · line 698

Quoted 4 lines
drivers/nvme/host/fabrics.c +1 -0
@@ -695,6 +695,7 @@ static const match_table_t opt_tokens = {
 	{ NVMF_OPT_NR_WRITE_QUEUES,	"nr_write_queues=%d"	},
 	{ NVMF_OPT_NR_POLL_QUEUES,	"nr_poll_queues=%d"	},
 	{ NVMF_OPT_TOS,			"tos=%d"		},
+	{ NVMF_OPT_TCLASS,		"tclass=%d"		},

Do you really need new UAPI for v6? Any reason not to apply existing NVMF_OPT_TOS to ipv6 sockets?

author reviewer
#6 lore raw

Re: [PATCH v2 2/3] nvme-tcp: support IPv6 traffic class

Hi Stanislav,

Quoted 38 lines · Stanislav Fomichev
Quoted 37 lines · Geliang Tang

From: Geliang Tang <[email protected]>

The NVMe/TCP transport needs to configure the IPv6 traffic class on its
queue sockets, but the fabrics option parser currently has no corresponding
option.

Add the tclass option to the fabrics parser and store the validated value
in struct nvmf_ctrl_options. Negative values are rejected, while values
greater than 255 are clamped to 255, matching the valid range of the IPv6
traffic class field.

Keep -1 as the default value to let the transport use its default traffic
class.

The fabrics layer now supports parsing the tclass option, but the NVMe/TCP
transport does not currently advertise or apply it.

Allow NVMe/TCP to use NVMF_OPT_TCLASS and set IPV6_TCLASS on each queue
socket via ip6_sock_set_tclass() when a traffic class is specified, keeping
the existing IPv4 TOS handling unchanged.

---
 drivers/nvme/host/fabrics.c | 18 ++++++++++++++++++ drivers/nvme/host/fabrics.h |  3 +++ drivers/nvme/host/tcp.c     |  8 +++++++-

 3 files changed, 28 insertions(+), 1 deletion(-)
Quoted 14 lines

drivers/nvme/host/fabrics.c static const match_table_t opt_tokens = { · line 699

Quoted 11 lines
drivers/nvme/host/fabrics.c +1 -0
index ac3d4f400601..643c03dc7bcb 100644
@@ -695,6 +695,7 @@ static const match_table_t opt_tokens = {
  	{ NVMF_OPT_NR_WRITE_QUEUES,	"nr_write_queues=%d"	},
  	{ NVMF_OPT_NR_POLL_QUEUES,	"nr_poll_queues=%d"	},
  	{
 NVMF_OPT_TOS,			"tos=%d"		},
+	{ NVMF_OPT_TCLASS,		"tclass=%d"		},

Do you really need new UAPI for v6? Any reason not to apply existing NVMF_OPT_TOS to ipv6 sockets?

Thanks for the feedback. I've given this careful consideration, and you're absolutely right - there's no strong reason to introduce new UAPI specifically for IPv6 when the existing NVMF_OPT_TOS can be reused. I'll reuse it for IPv6 in v3 and drop this new "tclass" option.

Thanks,
-Geliang

author reviewer
#7 lore raw

PATCH v2 3/3 nvmet-tcp: support IPv6 traffic class

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

nvmet-tcp currently applies the received IPv4 TOS value when setting up a queue socket, but does not handle the IPv6 traffic class.

Extend the queue socket setup to handle AF_INET6 sockets. Obtain the traffic class from the IPv6 socket's rcv_flowinfo and apply it through ip6_sock_set_tclass(). Keep the existing IPv4 TOS handling unchanged.

---
 drivers/nvme/target/tcp.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/nvme/target/tcp.c +8 -0
index 75a276d73be3..1d65519ebc0f 100644
@@ -1725,6 +1725,14 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 	/* Set socket type of service */
 	if (inet->rcv_tos > 0)
 		ip_sock_set_tos(sock->sk, inet->rcv_tos);
+#if IS_ENABLED(CONFIG_IPV6)
+	if (sock->sk->sk_family == AF_INET6) {
+		u8 rcv_tclass = ip6_tclass(inet6_sk(sock->sk)->rcv_flowinfo);
+
+		if (rcv_tclass > 0)
+			ip6_sock_set_tclass(sock->sk, rcv_tclass);
+	}
+#endif
 
 	ret = 0;
 	write_lock_bh(&sock->sk->sk_callback_lock);
Signature · 1 line
2.53.0
#8 lore raw
In reply to Geliang Tang

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/31245611809

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/61eb4ed8fb16
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1142555

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)