Skip to contentSkip to conversation

nvme-tcp: add IPv6 traffic class support

PATCH Superseded Geliang Tang 1 participant 4 messages Jul 27

Subsystems (2)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover nvme-tcp: add IPv6 traffic class support

From: Geliang Tang <[email protected]>

This series adds IPv6 traffic class (tclass) support to the NVMe/TCP fabrics stack, mirroring the existing IPv4 TOS configuration.

The current code path only handles IPv4: nvme-fabrics exposes a 'tos' option, the host (nvme-tcp) applies IP_TOS to each queue socket, and the target (nvmet-tcp) reflects the value received on the wire. This leaves no way to control or observe the IPv6 traffic class for NVMe/TCP connections, which is a real gap for deployments running over IPv6 or MPTCP subflows that need a non-default tclass.

The series is split into three patches following the natural fabrics/host/target layering:

1. nvme-fabrics: add IPv6 traffic class option
Parse the new 'tclass' option in nvmf_parse_options(), validate that it is in the 0-255 range, and store it in struct nvmf_ctrl_options. Use -1 to mean "transport default", matching the existing tos handling.

2. nvme-tcp: support IPv6 traffic class
Advertise NVMF_OPT_TCLASS in the nvme_tcp transport and, when a tclass was provided, set IPV6_TCLASS on each queue socket. The existing IP_TOS path is left untouched so IPv4 behaviour is unchanged.

3. nvmet-tcp: support IPv6 traffic class
On the target side, differentiate the socket family: for AF_INET keep the existing TOS handling, and for AF_INET6 read the traffic class from the incoming socket and apply it via IPV6_TCLASS.

Follow-up changes outside of this series expose the same option through libnvme and the nvme-cli, so the new tclass can be selected from the user-facing command line; those patches will be sent in a separate series.

This was tested with the selftests mptcp_nvme.sh [1], extended in "nvme: add tclass support" to pass "--tclass" on the host side and validate that the value is reflected on the wire by the target.

[1]
https://patchwork.kernel.org/project/linux-nvme/cover/[email protected]/

Geliang Tang (3):
nvme-fabrics: add IPv6 traffic class option
nvme-tcp: support IPv6 traffic class
nvmet-tcp: support IPv6 traffic class

 drivers/nvme/host/fabrics.c | 18 ++++++++++++++++++
 drivers/nvme/host/fabrics.h |  3 +++
 drivers/nvme/host/tcp.c     |  8 +++++++-
 drivers/nvme/target/tcp.c   | 15 +++++++++++++--
 4 files changed, 41 insertions(+), 3 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/3 nvme-fabrics: add IPv6 traffic class option

From: Geliang Tang <[email protected]>

The NVMe/TCP transport needs to configure the IPv6 traffic class on its queue sockets, but the fabrics option parser currently has no corresponding option.

Add the tclass option to the fabrics parser and store the validated value in struct nvmf_ctrl_options. Negative values are rejected, while values greater than 255 are clamped to 255, matching the valid range of the IPv6 traffic class field.

Keep -1 as the default value to let the transport use its default traffic class.

---
 drivers/nvme/host/fabrics.c | 18 ++++++++++++++++++
 drivers/nvme/host/fabrics.h |  3 +++
 2 files changed, 21 insertions(+)
drivers/nvme/host/fabrics.c +18 -0
index ac3d4f400601..643c03dc7bcb 100644
@@ -695,6 +695,7 @@ static const match_table_t opt_tokens = {
 	{ NVMF_OPT_NR_WRITE_QUEUES,	"nr_write_queues=%d"	},
 	{ NVMF_OPT_NR_POLL_QUEUES,	"nr_poll_queues=%d"	},
 	{ NVMF_OPT_TOS,			"tos=%d"		},
+	{ NVMF_OPT_TCLASS,		"tclass=%d"		},
 #ifdef CONFIG_NVME_TCP_TLS
 	{ NVMF_OPT_KEYRING,		"keyring=%d"		},
 	{ NVMF_OPT_TLS_KEY,		"tls_key=%d"		},
@@ -734,6 +735,7 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
 	opts->hdr_digest = false;
 	opts->data_digest = false;
 	opts->tos = -1; /* < 0 == use transport default */
+	opts->tclass = -1; /* < 0 == use transport default */
 	opts->tls = false;
 	opts->tls_key = NULL;
 	opts->keyring = NULL;
@@ -991,6 +993,22 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
 			}
 			opts->tos = token;
 			break;
+		case NVMF_OPT_TCLASS:
+			if (match_int(args, &token)) {
+				ret = -EINVAL;
+				goto out;
+			}
+			if (token < 0) {
+				pr_err("Invalid traffic class %d\n", token);
+				ret = -EINVAL;
+				goto out;
+			}
+			if (token > 255) {
+				pr_warn("Clamping traffic class to 255\n");
+				token = 255;
+			}
+			opts->tclass = token;
+			break;
 		case NVMF_OPT_KEYRING:
 			if (match_int(args, &key_id) || key_id <= 0) {
 				ret = -EINVAL;
drivers/nvme/host/fabrics.h +3 -0
index caf5503d0833..3dfd40bd8960 100644
@@ -67,6 +67,7 @@ enum {
 	NVMF_OPT_KEYRING	= 1 << 26,
 	NVMF_OPT_TLS_KEY	= 1 << 27,
 	NVMF_OPT_CONCAT		= 1 << 28,
+	NVMF_OPT_TCLASS		= 1 << 29,
 };
 
 /**
@@ -109,6 +110,7 @@ enum {
  * @nr_write_queues: number of queues for write I/O
  * @nr_poll_queues: number of queues for polling I/O
  * @tos: type of service
+ * @tclass: IPv6 traffic class
  * @fast_io_fail_tmo: Fast I/O fail timeout in seconds
  */
 struct nvmf_ctrl_options {
@@ -139,6 +141,7 @@ struct nvmf_ctrl_options {
 	unsigned int		nr_write_queues;
 	unsigned int		nr_poll_queues;
 	int			tos;
+	int			tclass;
 	int			fast_io_fail_tmo;
 };
 
Signature · 1 line
2.53.0
#2 lore raw

PATCH 2/3 nvme-tcp: support IPv6 traffic class

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

The fabrics layer now supports parsing the tclass option, but the NVMe/TCP transport does not currently advertise or apply it.

Allow NVMe/TCP to use NVMF_OPT_TCLASS and set IPV6_TCLASS on each queue socket when a traffic class is specified. Keep the existing IPv4 TOS handling unchanged.

---
 drivers/nvme/host/tcp.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/nvme/host/tcp.c +7 -1
index ba5c7b3e2a7c..f35c047ea02b 100644
@@ -1779,6 +1779,7 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 {
 	struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
 	struct nvme_tcp_queue *queue = &ctrl->queues[qid];
+	int tclass = nctrl->opts->tclass;
 	int ret, rcv_pdu_size;
 	struct file *sock_file;
 
@@ -1837,6 +1838,10 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 	/* Set socket type of service */
 	if (nctrl->opts->tos >= 0)
 		ip_sock_set_tos(queue->sock->sk, nctrl->opts->tos);
+	if (tclass >= 0)
+		sock_common_setsockopt(queue->sock->sk->sk_socket, SOL_IPV6,
+				       IPV6_TCLASS, KERNEL_SOCKPTR(&tclass),
+				       sizeof(tclass));
 
 	/* Set 10 seconds timeout for icresp recvmsg */
 	queue->sock->sk->sk_rcvtimeo = 10 * HZ;
@@ -3041,7 +3046,8 @@ static struct nvmf_transport_ops nvme_tcp_transport = {
 			  NVMF_OPT_HOST_TRADDR | NVMF_OPT_CTRL_LOSS_TMO |
 			  NVMF_OPT_HDR_DIGEST | NVMF_OPT_DATA_DIGEST |
 			  NVMF_OPT_NR_WRITE_QUEUES | NVMF_OPT_NR_POLL_QUEUES |
-			  NVMF_OPT_TOS | NVMF_OPT_HOST_IFACE | NVMF_OPT_TLS |
+			  NVMF_OPT_TOS | NVMF_OPT_TCLASS |
+			  NVMF_OPT_HOST_IFACE | NVMF_OPT_TLS |
 			  NVMF_OPT_KEYRING | NVMF_OPT_TLS_KEY | NVMF_OPT_CONCAT,
 	.create_ctrl	= nvme_tcp_create_ctrl,
 };
Signature · 1 line
2.53.0
#3 lore raw

PATCH 3/3 nvmet-tcp: support IPv6 traffic class

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

nvmet-tcp currently applies the received IPv4 TOS value when setting up a queue socket, but does not handle the IPv6 traffic class.

Extend the queue socket setup to handle AF_INET6 sockets. Obtain the traffic class from the IPv6 socket and apply it through IPV6_TCLASS. Keep the existing IPv4 TOS handling unchanged.

---
 drivers/nvme/target/tcp.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
drivers/nvme/target/tcp.c +13 -2
index 75a276d73be3..fcdc73dfe5b0 100644
@@ -13,6 +13,7 @@
 #include <linux/nvme-keyring.h>
 #include <net/sock.h>
 #include <net/tcp.h>
+#include <net/ipv6.h>
 #include <net/tls.h>
 #include <net/tls_prot.h>
 #include <net/handshake.h>
@@ -1723,8 +1724,18 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 		sock_set_priority(sock->sk, so_priority);
 
 	/* Set socket type of service */
-	if (inet->rcv_tos > 0)
-		ip_sock_set_tos(sock->sk, inet->rcv_tos);
+	if (sock->sk->sk_family == AF_INET) {
+		if (inet->rcv_tos > 0)
+			ip_sock_set_tos(sock->sk, inet->rcv_tos);
+	} else if (sock->sk->sk_family == AF_INET6) {
+		int tclass = inet6_sk(sock->sk)->tclass;
+
+		if (tclass > 0)
+			sock_common_setsockopt(sock->sk->sk_socket, SOL_IPV6,
+					       IPV6_TCLASS,
+					       KERNEL_SOCKPTR(&tclass),
+					       sizeof(tclass));
+	}
 
 	ret = 0;
 	write_lock_bh(&sock->sk->sk_callback_lock);
Signature · 1 line
2.53.0