Skip to contentSkip to conversation

nvme-tcp: add IPv6 traffic class support

PATCH In review Geliang Tang 7 participants 13 messages Aug 18 – Oct 16

Subsystems (5)
lore
Share & tools
lore

Geliang Tang

author reviewer
#0 lore raw

v4 cover nvme-tcp: add IPv6 traffic class support

From: Geliang Tang <[email protected]>

This series adds IPv6 traffic class (tclass) support to the NVMe/TCP fabrics stack, mirroring the existing IPv4 TOS configuration.

The current code path only handles IPv4: nvme-fabrics exposes a 'tos' option, the host (nvme-tcp) applies IP_TOS to each queue socket, and the target (nvmet-tcp) reflects the value received on the wire. This leaves no way to control or observe the IPv6 traffic class for NVMe/TCP connections, which is a real gap for deployments running over IPv6 that need a non-default tclass.

The series also refactors the socket option configuration on both host and target sides to use the generic do_sock_setsockopt() helper. This change eliminates the need for protocol-specific helpers and decouples the configuration from the underlying transport protocol, laying the groundwork for future protocol extensions such as MPTCP.

The series is split into two logical parts: the do_sock_setsockopt refactoring (patches 1-2) and the IPv6 traffic class support (patches 3-4), following the natural host/target layering.

This was tested with the selftests mptcp_nvme.sh [1] to pass "--tos" on the host side and validate that the value is reflected on the wire by the target.

[1]
https://patchwork.kernel.org/project/linux-nvme/cover/[email protected]/

v4:
- check return values of do_sock_setsockopt, as suggested by Sashiko.
- change "u8 tclass" to "int tclass" in nvmet_tcp_sock_set_tclass, as suggested by Sashiko.
- change "u8 tos" to "int tos" in nvmet_tcp_sock_set_tos.

v3:
- add two patches to unify socket option configuration using
do_sock_setsockopt on both host and target sides.
- use do_sock_setsockopt for IPV6_TCLASS rather than exporting and
calling ip6_sock_set_tclass, as suggested by Jakub.
- reuse the existing NVMF_OPT_TOS for IPv6 sockets instead of adding
a new NVMF_OPT_TCLASS option, as suggested by Stanislav.
- https://patchwork.kernel.org/project/linux-nvme/cover/[email protected]/

v2:
- Export and use the ip6_sock_set_tclass() helper in both target and host
paths, aligning with the existing ip_sock_set_tos() usage.
- Add CONFIG_IPV6 and AF_INET6 guards to avoid build failures when IPv6
is disabled.
- Use rcv_flowinfo from the target socket instead of np->tclass.
- https://patchwork.kernel.org/project/linux-nvme/cover/[email protected]/

v1:
- https://patchwork.kernel.org/project/linux-nvme/cover/[email protected]/

Geliang Tang (4):
nvmet-tcp: unify sockopt with do_sock_setsockopt
nvme-tcp: unify sockopt with do_sock_setsockopt
nvmet-tcp: support IPv6 traffic class
nvme-tcp: support IPv6 traffic class

 drivers/nvme/host/tcp.c   | 111 ++++++++++++++++++++++++++++++++++----
 drivers/nvme/target/tcp.c | 102 +++++++++++++++++++++++++++++++----
 2 files changed, 193 insertions(+), 20 deletions(-)
Signature · 1 line
2.53.0
author reviewer
#1 lore raw

PATCH v4 1/4 nvmet-tcp: unify sockopt with do_sock_setsockopt

From: Geliang Tang <[email protected]>

This patch consolidates socket option settings in nvmet-tcp by utilizing the generic do_sock_setsockopt() helper for options including SO_LINGER, SO_PRIORITY, SO_REUSEADDR, TCP_NODELAY, and IP_TOS. This change eliminates the need to export and use specialized helpers for each individual socket option.

A key benefit of this refactoring is that it decouples the socket option configuration from the underlying transport protocol. This makes it easier to extend nvmet-tcp to support other protocols, such as MPTCP, in the future, as do_sock_setsockopt() abstracts away protocol-specific differences without requiring per-option protocol-specific wrappers.

---
 drivers/nvme/target/tcp.c | 82 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 72 insertions(+), 10 deletions(-)
drivers/nvme/target/tcp.c +72 -10
index 75a276d73be3..38b0c446ae89 100644
@@ -1696,10 +1696,50 @@ static void nvmet_tcp_state_change(struct sock *sk)
 	read_unlock_bh(&sk->sk_callback_lock);
 }
 
+static int nvmet_tcp_sock_no_linger(struct sock *sk)
+{
+	struct linger ling = { .l_onoff = 1, .l_linger = 0 };
+
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET, SO_LINGER,
+				  KERNEL_SOCKPTR(&ling), sizeof(ling));
+}
+
+static int nvmet_tcp_sock_set_priority(struct sock *sk, u32 priority)
+{
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET, SO_PRIORITY,
+				  KERNEL_SOCKPTR(&priority), sizeof(priority));
+}
+
+static int nvmet_tcp_sock_set_reuseaddr(struct sock *sk)
+{
+	int val = SK_CAN_REUSE;
+
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET,
+				  SO_REUSEADDR, KERNEL_SOCKPTR(&val),
+				  sizeof(val));
+}
+
+static int nvmet_tcp_sock_set_nodelay(struct sock *sk)
+{
+	int val = 1;
+
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_TCP, TCP_NODELAY,
+				  KERNEL_SOCKPTR(&val), sizeof(val));
+}
+
+static int nvmet_tcp_sock_set_tos(struct sock *sk)
+{
+	int tos = inet_sk(sk)->rcv_tos;
+
+	if (tos > 0)
+		return do_sock_setsockopt(sk->sk_socket, false, SOL_IP, IP_TOS,
+					  KERNEL_SOCKPTR(&tos), sizeof(tos));
+	return 0;
+}
+
 static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 {
 	struct socket *sock = queue->sock;
-	struct inet_sock *inet = inet_sk(sock->sk);
 	int ret;
 
 	ret = kernel_getsockname(sock,
@@ -1717,14 +1757,20 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 	 * close. This is done to prevent stale data from being sent should
 	 * the network connection be restored before TCP times out.
 	 */
-	sock_no_linger(sock->sk);
+	ret = nvmet_tcp_sock_no_linger(sock->sk);
+	if (ret)
+		return ret;
 
-	if (so_priority > 0)
-		sock_set_priority(sock->sk, so_priority);
+	if (so_priority > 0) {
+		ret = nvmet_tcp_sock_set_priority(sock->sk, so_priority);
+		if (ret)
+			return ret;
+	}
 
 	/* Set socket type of service */
-	if (inet->rcv_tos > 0)
-		ip_sock_set_tos(sock->sk, inet->rcv_tos);
+	ret = nvmet_tcp_sock_set_tos(sock->sk);
+	if (ret)
+		return ret;
 
 	ret = 0;
 	write_lock_bh(&sock->sk->sk_callback_lock);
@@ -2098,10 +2144,26 @@ static int nvmet_tcp_add_port(struct nvmet_port *nport)
 	port->sock->sk->sk_user_data = port;
 	port->data_ready = port->sock->sk->sk_data_ready;
 	port->sock->sk->sk_data_ready = nvmet_tcp_listen_data_ready;
-	sock_set_reuseaddr(port->sock->sk);
-	tcp_sock_set_nodelay(port->sock->sk);
-	if (so_priority > 0)
-		sock_set_priority(port->sock->sk, so_priority);
+	ret = nvmet_tcp_sock_set_reuseaddr(port->sock->sk);
+	if (ret) {
+		pr_err("failed to set SO_REUSEADDR on port socket %d\n", ret);
+		goto err_sock;
+	}
+
+	ret = nvmet_tcp_sock_set_nodelay(port->sock->sk);
+	if (ret) {
+		pr_err("failed to set TCP_NODELAY on port socket %d\n", ret);
+		goto err_sock;
+	}
+
+	if (so_priority > 0) {
+		ret = nvmet_tcp_sock_set_priority(port->sock->sk, so_priority);
+		if (ret) {
+			pr_err("failed to set SO_PRIORITY on port socket %d\n",
+			       ret);
+			goto err_sock;
+		}
+	}
 
 	ret = kernel_bind(port->sock, (struct sockaddr_unsized *)&port->addr,
 			sizeof(port->addr));
Signature · 1 line
2.53.0
Reviewed-by on v4 1/4
#2 lore raw

Re: [PATCH v4 1/4] nvmet-tcp: unify sockopt with do_sock_setsockopt

Quoted 14 lines · Geliang Tang

From: Geliang Tang <[email protected]>

This patch consolidates socket option settings in nvmet-tcp by utilizing the generic do_sock_setsockopt() helper for options including SO_LINGER, SO_PRIORITY, SO_REUSEADDR, TCP_NODELAY, and IP_TOS. This change eliminates the need to export and use specialized helpers for each individual socket option.

A key benefit of this refactoring is that it decouples the socket option configuration from the underlying transport protocol. This makes it easier to extend nvmet-tcp to support other protocols, such as MPTCP, in the future, as do_sock_setsockopt() abstracts away protocol-specific differences without requiring per-option protocol-specific wrappers.

Quoted 5 lines
---
  drivers/nvme/target/tcp.c | 82 ++++++++++++++++++++++++++++++++++-----
  1 file changed, 72 insertions(+), 10 deletions(-)

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke                  Kernel Storage Architect
[email protected]                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
#3 lore raw

Re: [PATCH v4 1/4] nvmet-tcp: unify sockopt with do_sock_setsockopt

In reply to Geliang Tang
Quoted 7 lines · Geliang Tang

From: Geliang Tang <[email protected]>

This patch consolidates socket option settings in nvmet-tcp by utilizing the generic do_sock_setsockopt() helper for options including SO_LINGER, SO_PRIORITY, SO_REUSEADDR, TCP_NODELAY, and IP_TOS. This change eliminates the need to export and use specialized helpers for each individual socket option.

Hmm. I see that do_sock_setsockopt is exported, but it really should not be. It's really just an internal helper exposed for io_uring and no modular code should be using it (never mind the non-GPL export).

Quoted 5 lines

A key benefit of this refactoring is that it decouples the socket option configuration from the underlying transport protocol. This makes it easier to extend nvmet-tcp to support other protocols, such as MPTCP, in the future, as do_sock_setsockopt() abstracts away protocol-specific differences without requiring per-option protocol-specific wrappers.

We really should have generic helpers in the networking code for this and not duplicate them in driver using socket options.

#4 lore raw

Re: [PATCH v4 1/4] nvmet-tcp: unify sockopt with do_sock_setsockopt

Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Geliang Tang

From: Geliang Tang <[email protected]>

This patch consolidates socket option settings in nvmet-tcp by utilizing the generic do_sock_setsockopt() helper for options including SO_LINGER, SO_PRIORITY, SO_REUSEADDR, TCP_NODELAY, and IP_TOS. This change eliminates the need to export and use specialized helpers for each individual socket option.

Quoted 3 lines

Hmm. I see that do_sock_setsockopt is exported, but it really should not be. It's really just an internal helper exposed for io_uring and no modular code should be using it (never mind the non-GPL export).

Agreed. I am happy to remove the export, given it was introduced by my change in commit 1406245c2945 ("net/socket: Break down __sys_setsockopt")

#5 lore raw

Re: [PATCH v4 1/4] nvmet-tcp: unify sockopt with do_sock_setsockopt

Quoted 5 lines · Breno Leitao

On Wed, Aug 19, 2026 at 11:04:39AM +0200, Christoph Hellwig wrote:

Agreed. I am happy to remove the export, given it was introduced by my change in commit 1406245c2945 ("net/socket: Break down __sys_setsockopt")

Patch sent to net:

https://lore.kernel.org/all/[email protected]/

maintainer
#6 lore raw

Re: [PATCH v4 1/4] nvmet-tcp: unify sockopt with do_sock_setsockopt

In reply to Christoph Hellwig
Quoted 8 lines · On Wed, 19 Aug 2026 11:04:39 +0200 Christoph Hellwig wrote:
Quoted 5 lines

A key benefit of this refactoring is that it decouples the socket option configuration from the underlying transport protocol. This makes it easier to extend nvmet-tcp to support other protocols, such as MPTCP, in the future, as do_sock_setsockopt() abstracts away protocol-specific differences without requiring per-option protocol-specific wrappers.

We really should have generic helpers in the networking code for this and not duplicate them in driver using socket options.

What do you mean by "for this" ?

author reviewer
#7 lore raw

Re: [PATCH v4 1/4] nvmet-tcp: unify sockopt with do_sock_setsockopt

In reply to Christoph Hellwig

Hi,

Sorry for the late reply. In another patch "mptcp: sockopt: implement IPV6_TCLASS" [1] on the MPTCP mailing list, I implemented the __ip6_sock_set_tclass() helper and hoped it would be upstreamed first, but it is still pending.

Thank you all very much for your review comments, but I was caught in a dilemma after receiving different suggestions.

Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Geliang Tang

From: Geliang Tang <[email protected]>

This patch consolidates socket option settings in nvmet-tcp by utilizing
the generic do_sock_setsockopt() helper for options including SO_LINGER,
SO_PRIORITY, SO_REUSEADDR, TCP_NODELAY, and IP_TOS. This change eliminates
the need to export and use specialized helpers for each individual socket
option.

Quoted 4 lines

Hmm.  I see that do_sock_setsockopt is exported, but it really should not be.  It's really just an internal helper exposed for io_uring and no modular code should be using it (never mind the non-GPL export).

You are right, my current version has several problems:

1. do_sock_setsockopt() should be exported with EXPORT_SYMBOL_GPL(), not EXPORT_SYMBOL().

2. The "compat" parameter of do_sock_setsockopt() should be "true" instead of "false" to skip BPF cgroup setsockopt programs, matching the behavior of the original nvme-tcp code.

3. The helpers should return "void" instead of "int", to preserve the original behavior in nvme-tcp where these socket options are set on a best-effort basis.

4. Error logging (pr_err) should be inside the helpers to minimize changes to nvme-tcp code.

5. The helpers should not be duplicated between host and target, they should be consolidated into a shared header, e.g., nvme/common/tcp- sockopt.h.

I wonder if we can move forward with this approach once these issues are fixed?

Quoted 11 lines
Quoted 10 lines

A key benefit of this refactoring is that it decouples the socket option
configuration from the underlying transport protocol. This makes it easier to extend nvmet-tcp to support other protocols, such as MPTCP, in
the future, as do_sock_setsockopt() abstracts away protocol- specific
differences without requiring per-option protocol-specific wrappers.

Quoted 2 lines

We really should have generic helpers in the networking code for this and not duplicate them in driver using socket options.

However, I still need to use do_sock_setsockopt() for a fundamental
reason: the upcoming NVMe MPTCP support series requires protocol- independent socket option handling.

When NVMe creates MPTCP sockets instead of TCP sockets, calling protocol-specific helpers like tcp_sock_set_nodelay() or ip_sock_set_tos() directly will not work correctly on MPTCP sockets. MPTCP needs to set options not only on the MPTCP socket itself but also synchronize them to all subflow sockets.

In the previous NVMe MPTCP support implementation [2], I exported protocol-specific sockopt helpers separately on the TCP side and MPTCP side, then had NVMe switch between them based on the protocol. This is not as clean as using do_sock_setsockopt() for protocol-independent calls.

For MPTCP, we could use sock_common_setsockopt() which always dispatches to the protocol's .setsockopt for all options. However, for TCP, this doesn't work correctly for SOL_SOCKET level options (SO_LINGER, SO_PRIORITY, SO_REUSEADDR, SO_BINDTODEVICE) which should be handled by sock_setsockopt() directly. That's why do_sock_setsockopt() is used here - it intelligently chooses between sock_setsockopt() for SOL_SOCKET options and sock->ops->setsockopt() for protocol-specific options, making it work correctly for both TCP and MPTCP without requiring nvme-tcp to know which protocol it is dealing with. This is the key to enabling NVMe MPTCP support without adding MPTCP-specific exports or protocol dispatch logic in nvme-tcp. A more detailed discussion can be found in [3].

I would appreciate any further suggestions you may have.

Thanks,
-Geliang

[1]
https://patchwork.kernel.org/project/mptcp/patch/c17925172fd53c98f0ec5c0095c86e6e28cbebf4.1786938892.git.tanggeliang@kylinos.cn/
[2]
https://patchwork.kernel.org/project/linux-nvme/cover/[email protected]/
[3]
https://patchwork.kernel.org/project/linux-nvme/patch/a1fa5dd3ebb0cafe1bf7948bf2a96436e9452e19.1779934709.git.tanggeliang@kylinos.cn/

author reviewer
#8 lore raw

PATCH v4 2/4 nvme-tcp: unify sockopt with do_sock_setsockopt

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

This patch consolidates socket option settings in nvme-tcp by utilizing the generic do_sock_setsockopt() helper for options including SO_LINGER, SO_PRIORITY, TCP_NODELAY, IP_TOS, SO_BINDTODEVICE, and TCP_SYNCNT.

Compared to the target-side implementation, this patch additionally converts SO_BINDTODEVICE and TCP_SYNCNT to use the same unified mechanism. This change eliminates the need to export and use specialized helpers for each individual socket option.

---
 drivers/nvme/host/tcp.c | 91 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 81 insertions(+), 10 deletions(-)
drivers/nvme/host/tcp.c +81 -10
index ba5c7b3e2a7c..025cade370b2 100644
@@ -1774,6 +1774,47 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
 	return ret;
 }
 
+static int nvme_tcp_sock_no_linger(struct sock *sk)
+{
+	struct linger ling = { .l_onoff = 1, .l_linger = 0 };
+
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET, SO_LINGER,
+				  KERNEL_SOCKPTR(&ling), sizeof(ling));
+}
+
+static int nvme_tcp_sock_set_priority(struct sock *sk, u32 priority)
+{
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET, SO_PRIORITY,
+				  KERNEL_SOCKPTR(&priority), sizeof(priority));
+}
+
+static int nvme_tcp_sock_set_bindtodevice(struct sock *sk, char *iface)
+{
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET,
+				  SO_BINDTODEVICE, KERNEL_SOCKPTR(iface),
+				  strlen(iface));
+}
+
+static int nvme_tcp_sock_set_nodelay(struct sock *sk)
+{
+	int val = 1;
+
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_TCP, TCP_NODELAY,
+				  KERNEL_SOCKPTR(&val), sizeof(val));
+}
+
+static int nvme_tcp_sock_set_syncnt(struct sock *sk, int val)
+{
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_TCP, TCP_SYNCNT,
+				  KERNEL_SOCKPTR(&val), sizeof(val));
+}
+
+static int nvme_tcp_sock_set_tos(struct sock *sk, int tos)
+{
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_IP, IP_TOS,
+				  KERNEL_SOCKPTR(&tos), sizeof(tos));
+}
+
 static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 				key_serial_t pskid)
 {
@@ -1819,24 +1860,56 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 #endif
 
 	/* Single syn retry */
-	tcp_sock_set_syncnt(queue->sock->sk, 1);
+	ret = nvme_tcp_sock_set_syncnt(queue->sock->sk, 1);
+	if (ret) {
+		dev_err(nctrl->device,
+			"failed to set TCP_SYNCNT on queue %d err %d\n",
+			qid, ret);
+		goto err_sock;
+	}
 
 	/* Set TCP no delay */
-	tcp_sock_set_nodelay(queue->sock->sk);
+	ret = nvme_tcp_sock_set_nodelay(queue->sock->sk);
+	if (ret) {
+		dev_err(nctrl->device,
+			"failed to set TCP_NODELAY on queue %d err %d\n",
+			qid, ret);
+		goto err_sock;
+	}
 
 	/*
 	 * Cleanup whatever is sitting in the TCP transmit queue on socket
 	 * close. This is done to prevent stale data from being sent should
 	 * the network connection be restored before TCP times out.
 	 */
-	sock_no_linger(queue->sock->sk);
+	ret = nvme_tcp_sock_no_linger(queue->sock->sk);
+	if (ret) {
+		dev_err(nctrl->device,
+			"failed to set SO_LINGER on queue %d err %d\n",
+			qid, ret);
+		goto err_sock;
+	}
 
-	if (so_priority > 0)
-		sock_set_priority(queue->sock->sk, so_priority);
+	if (so_priority > 0) {
+		ret = nvme_tcp_sock_set_priority(queue->sock->sk, so_priority);
+		if (ret) {
+			dev_err(nctrl->device,
+				"failed to set SO_PRIORITY on queue %d err %d\n",
+				qid, ret);
+			goto err_sock;
+		}
+	}
 
 	/* Set socket type of service */
-	if (nctrl->opts->tos >= 0)
-		ip_sock_set_tos(queue->sock->sk, nctrl->opts->tos);
+	if (nctrl->opts->tos >= 0) {
+		ret = nvme_tcp_sock_set_tos(queue->sock->sk, nctrl->opts->tos);
+		if (ret) {
+			dev_err(nctrl->device,
+				"failed to set IP_TOS on queue %d err %d\n",
+				qid, ret);
+			goto err_sock;
+		}
+	}
 
 	/* Set 10 seconds timeout for icresp recvmsg */
 	queue->sock->sk->sk_rcvtimeo = 10 * HZ;
@@ -1864,10 +1937,8 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 
 	if (nctrl->opts->mask & NVMF_OPT_HOST_IFACE) {
 		char *iface = nctrl->opts->host_iface;
-		sockptr_t optval = KERNEL_SOCKPTR(iface);
 
-		ret = sock_setsockopt(queue->sock, SOL_SOCKET, SO_BINDTODEVICE,
-				      optval, strlen(iface));
+		ret = nvme_tcp_sock_set_bindtodevice(queue->sock->sk, iface);
 		if (ret) {
 			dev_err(nctrl->device,
 			  "failed to bind to interface %s queue %d err %d\n",
Signature · 1 line
2.53.0
author reviewer
#9 lore raw

PATCH v4 3/4 nvmet-tcp: support IPv6 traffic class

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Currently, nvmet-tcp only applies the received IPv4 TOS value when setting up a queue socket, but does not handle the IPv6 traffic class.

Extend the queue socket setup to handle AF_INET6 sockets. Obtain the traffic class from the IPv6 socket's rcv_flowinfo and apply it through IPV6_TCLASS using do_sock_setsockopt().

---
 drivers/nvme/target/tcp.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
drivers/nvme/target/tcp.c +20 -0
index 38b0c446ae89..ed413aa5601c 100644
@@ -1737,6 +1737,22 @@ static int nvmet_tcp_sock_set_tos(struct sock *sk)
 	return 0;
 }
 
+static int nvmet_tcp_sock_set_tclass(struct sock *sk)
+{
+#if IS_ENABLED(CONFIG_IPV6)
+	if (sk->sk_family == AF_INET6) {
+		int tclass = ip6_tclass(inet6_sk(sk)->rcv_flowinfo);
+
+		if (tclass > 0)
+			return do_sock_setsockopt(sk->sk_socket, false,
+						  SOL_IPV6, IPV6_TCLASS,
+						  KERNEL_SOCKPTR(&tclass),
+						  sizeof(tclass));
+	}
+#endif
+	return 0;
+}
+
 static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 {
 	struct socket *sock = queue->sock;
@@ -1772,6 +1788,10 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 	if (ret)
 		return ret;
 
+	ret = nvmet_tcp_sock_set_tclass(sock->sk);
+	if (ret)
+		return ret;
+
 	ret = 0;
 	write_lock_bh(&sock->sk->sk_callback_lock);
 	if (sock->sk->sk_state != TCP_ESTABLISHED) {
Signature · 1 line
2.53.0
author reviewer
#10 lore raw

PATCH v4 4/4 nvme-tcp: support IPv6 traffic class

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Currently, nvme-tcp host only supports setting the IPv4 TOS value when a TOS is specified, but does not handle the IPv6 traffic class.

Extend the queue socket setup to handle AF_INET6 sockets by applying the TOS value to both IPv4 and IPv6 sockets. For IPv6, the TOS value is set as the IPv6 traffic class via IPV6_TCLASS using do_sock_setsockopt().

---
 drivers/nvme/host/tcp.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
drivers/nvme/host/tcp.c +20 -0
index 025cade370b2..5f30be04a9dd 100644
@@ -1815,6 +1815,17 @@ static int nvme_tcp_sock_set_tos(struct sock *sk, int tos)
 				  KERNEL_SOCKPTR(&tos), sizeof(tos));
 }
 
+static int nvme_tcp_sock_set_tclass(struct sock *sk, int tclass)
+{
+#if IS_ENABLED(CONFIG_IPV6)
+	if (sk->sk_family == AF_INET6)
+		return do_sock_setsockopt(sk->sk_socket, false, SOL_IPV6,
+					  IPV6_TCLASS, KERNEL_SOCKPTR(&tclass),
+					  sizeof(tclass));
+#endif
+	return 0;
+}
+
 static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 				key_serial_t pskid)
 {
@@ -1909,6 +1920,15 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 				qid, ret);
 			goto err_sock;
 		}
+
+		ret = nvme_tcp_sock_set_tclass(queue->sock->sk,
+					       nctrl->opts->tos);
+		if (ret) {
+			dev_err(nctrl->device,
+				"failed to set IPV6_TCLASS on queue %d err %d\n",
+				qid, ret);
+			goto err_sock;
+		}
 	}
 
 	/* Set 10 seconds timeout for icresp recvmsg */
Signature · 1 line
2.53.0
maintainer
#11 lore raw

Re: [PATCH v4 4/4] nvme-tcp: support IPv6 traffic class

Quoted 37 lines · Geliang Tang

From: Geliang Tang <[email protected]>

Currently, nvme-tcp host only supports setting the IPv4 TOS value when a TOS is specified, but does not handle the IPv6 traffic class.

Extend the queue socket setup to handle AF_INET6 sockets by applying the TOS value to both IPv4 and IPv6 sockets. For IPv6, the TOS value is set as the IPv6 traffic class via IPV6_TCLASS using do_sock_setsockopt().

---
  drivers/nvme/host/tcp.c | 20 ++++++++++++++++++++
  1 file changed, 20 insertions(+)
drivers/nvme/host/tcp.c +11 -0
index 025cade370b2..5f30be04a9dd 100644
@@ -1815,6 +1815,17 @@ static int nvme_tcp_sock_set_tos(struct sock *sk, int tos)
  				  KERNEL_SOCKPTR(&tos), sizeof(tos));
  }
 
+static int nvme_tcp_sock_set_tclass(struct sock *sk, int tclass)
+{
+#if IS_ENABLED(CONFIG_IPV6)
+	if (sk->sk_family == AF_INET6)
+		return do_sock_setsockopt(sk->sk_socket, false, SOL_IPV6,
+					  IPV6_TCLASS, KERNEL_SOCKPTR(&tclass),
+					  sizeof(tclass));
+#endif
+	return 0;
+}
+
  static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
  				key_serial_t pskid)
  {

drivers/nvme/host/tcp.c static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid, · line 1925

Quoted 6 lines
drivers/nvme/host/tcp.c +3 -0
@@ -1909,6 +1920,15 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
  				qid, ret);
  			goto err_sock;
  		}
+
+		ret = nvme_tcp_sock_set_tclass(queue->sock->sk,
+					       nctrl->opts->tos);

Won't it just be cleaner to add a generic ip_sock_set_tclass() that hides the IS_ENABLED(CONFIG_IPV6) ifdef? Why do we need to wrap all these generic helpers?

#12 lore raw
In reply to Geliang Tang

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/32107026013

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/0547cdad1bc1
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1147565

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)