Skip to contentSkip to conversation

nvme/tcp: handle tls partially sent records in write_space()

PATCH Superseded Wilfred Mallawa 4 participants 5 messages Oct 10, 2025 NVMe

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 nvme/tcp: handle tls partially sent records in write_space()

From: Wilfred Mallawa <[email protected]>

With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for example, hitting an EAGAIN from tcp_sendmsg_locked(). This is not known to the NVMe TCP driver, as the TLS layer successfully generated a record.

Typically, the TLS write_space() callback would ensure such records are retried, but in the NVMe TCP Host driver, write_space() invokes nvme_tcp_write_space(). This causes a partially sent record in the TLS TX list to timeout after not being retried.

This patch fixes the above by calling queue->write_space(), which calls into the TLS layer to retry any pending records.

Fixes be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")

---
V2:
- Unconditionally invoke TLS write_space(). This means we don't need to export tls_is_partially_sent_record()
---
drivers/nvme/host/tcp.c | 5 +++++

 1 file changed, 5 insertions(+)
drivers/nvme/host/tcp.c +5 -0
index 1413788ca7d5..6016510577bd 100644
@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock *sk)
 
 	read_lock_bh(&sk->sk_callback_lock);
 	queue = sk->sk_user_data;
+
 	if (likely(queue && sk_stream_is_writeable(sk))) {
 		clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
+		/* Ensure pending TLS partial records are retried */
+		if (nvme_tcp_queue_tls(queue))
+			queue->write_space(sk);
+
 		queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
 	}
 	read_unlock_bh(&sk->sk_callback_lock);
Signature · 1 line
2.51.0
#1 lore raw
Quoted 30 lines · Wilfred Mallawa

From: Wilfred Mallawa <[email protected]>

With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for example, hitting an EAGAIN from tcp_sendmsg_locked(). This is not known to the NVMe TCP driver, as the TLS layer successfully generated a record.

Typically, the TLS write_space() callback would ensure such records are retried, but in the NVMe TCP Host driver, write_space() invokes nvme_tcp_write_space(). This causes a partially sent record in the TLS TX list to timeout after not being retried.

This patch fixes the above by calling queue->write_space(), which calls into the TLS layer to retry any pending records.

Fixes be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")

---
V2:
- Unconditionally invoke TLS write_space(). This means we don't need to export tls_is_partially_sent_record()
---
drivers/nvme/host/tcp.c | 5 +++++

 1 file changed, 5 insertions(+)
drivers/nvme/host/tcp.c +0 -0
index 1413788ca7d5..6016510577bd 100644

drivers/nvme/host/tcp.c static void nvme_tcp_write_space(struct sock *sk) · line 1082

Quoted 4 lines
drivers/nvme/host/tcp.c +1 -0
@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock *sk)
 
 	read_lock_bh(&sk->sk_callback_lock);
 	queue = sk->sk_user_data;
+

White space change... Remove please.

drivers/nvme/host/tcp.c static void nvme_tcp_write_space(struct sock *sk) · line 1088

Quoted 6 lines
drivers/nvme/host/tcp.c +4 -0
 	if (likely(queue && sk_stream_is_writeable(sk))) {
 		clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
+		/* Ensure pending TLS partial records are retried */
+		if (nvme_tcp_queue_tls(queue))
+			queue->write_space(sk);
+

Maybe remove this empty line too.

Quoted 3 lines
drivers/nvme/host/tcp.c +0 -0
 		queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
 	}
 	read_unlock_bh(&sk->sk_callback_lock);
Signature · 2 lines
Damien Le Moal
Western Digital Research
Reviewed-by on v2 1/1
#3 lore raw
In reply to Wilfred Mallawa
Quoted 33 lines · Wilfred Mallawa

From: Wilfred Mallawa <[email protected]>

With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for example, hitting an EAGAIN from tcp_sendmsg_locked(). This is not known to the NVMe TCP driver, as the TLS layer successfully generated a record.

Typically, the TLS write_space() callback would ensure such records are retried, but in the NVMe TCP Host driver, write_space() invokes nvme_tcp_write_space(). This causes a partially sent record in the TLS TX list to timeout after not being retried.

This patch fixes the above by calling queue->write_space(), which calls into the TLS layer to retry any pending records.

Fixes be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")

---
V2:
- Unconditionally invoke TLS write_space(). This means we don't need to export tls_is_partially_sent_record()
---
drivers/nvme/host/tcp.c | 5 +++++

  1 file changed, 5 insertions(+)
drivers/nvme/host/tcp.c +0 -0
index 1413788ca7d5..6016510577bd 100644
@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock *sk)
 
  	read_lock_bh(&sk->sk_callback_lock);
  	queue = sk->sk_user_data;

drivers/nvme/host/tcp.c static void nvme_tcp_write_space(struct sock *sk) · line 1091

Quoted 10 lines
drivers/nvme/host/tcp.c +5 -0
+
  	if (likely(queue && sk_stream_is_writeable(sk))) {
  		clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
+		/* Ensure pending TLS partial records are retried */
+		if (nvme_tcp_queue_tls(queue))
+			queue->write_space(sk);
+
  		queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
  	}
  	read_unlock_bh(&sk->sk_callback_lock);

Minus the whitespace change:

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke                  Kernel Storage Architect
[email protected]                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
#2 lore raw
In reply to Damien Le Moal
Quoted 46 lines · Damien Le Moal
Quoted 38 lines · Wilfred Mallawa

From: Wilfred Mallawa <[email protected]>

With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for
example, hitting an EAGAIN from tcp_sendmsg_locked(). This is not known
to the NVMe TCP driver, as the TLS layer successfully generated a record.

Typically, the TLS write_space() callback would ensure such records are
retried, but in the NVMe TCP Host driver, write_space() invokes nvme_tcp_write_space(). This causes a partially sent record in the TLS TX
list to timeout after not being retried.

This patch fixes the above by calling queue->write_space(), which calls
into the TLS layer to retry any pending records.

Fixes be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")

---
V2:
    - Unconditionally invoke TLS write_space(). This means we don't need
      to export tls_is_partially_sent_record()
---
drivers/nvme/host/tcp.c | 5 +++++

 1 file changed, 5 insertions(+)
drivers/nvme/host/tcp.c +0 -0
index 1413788ca7d5..6016510577bd 100644
@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock
 *sk)
Quoted 4 lines


  read_lock_bh(&sk->sk_callback_lock);
  queue = sk->sk_user_data;
+

White space change... Remove please.

Quoted 9 lines

drivers/nvme/host/tcp.c static void nvme_tcp_write_space(struct sock *sk) · line 1088

Quoted 7 lines

  if (likely(queue && sk_stream_is_writeable(sk))) { clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);

drivers/nvme/host/tcp.c +1 -0
+		/* Ensure pending TLS partial records are retried

*/

drivers/nvme/host/tcp.c +3 -0
+		if (nvme_tcp_queue_tls(queue))
+			queue->write_space(sk);
+

Maybe remove this empty line too.

Okay, Fixed in V3. Thanks!

Wilfred

Quoted 6 lines
Quoted 5 lines

  queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue-

Quoted 1 line

io_work);

  }
  read_unlock_bh(&sk->sk_callback_lock);

#4 lore raw
In reply to Hannes Reinecke
Quoted 51 lines · Hannes Reinecke
Quoted 50 lines · Wilfred Mallawa

From: Wilfred Mallawa <[email protected]>

With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for
example, hitting an EAGAIN from tcp_sendmsg_locked(). This is not known
to the NVMe TCP driver, as the TLS layer successfully generated a record.

Typically, the TLS write_space() callback would ensure such records are
retried, but in the NVMe TCP Host driver, write_space() invokes nvme_tcp_write_space(). This causes a partially sent record in the TLS TX
list to timeout after not being retried.

This patch fixes the above by calling queue->write_space(), which calls
into the TLS layer to retry any pending records.

Fixes be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")

---
V2:
     - Unconditionally invoke TLS write_space(). This means we don't need
       to export tls_is_partially_sent_record()
---
drivers/nvme/host/tcp.c | 5 +++++

  1 file changed, 5 insertions(+)
drivers/nvme/host/tcp.c +0 -0
index 1413788ca7d5..6016510577bd 100644
@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock
 *sk)


   read_lock_bh(&sk->sk_callback_lock);
   queue = sk->sk_user_data;
+
   if (likely(queue && sk_stream_is_writeable(sk))) { clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);

drivers/nvme/host/tcp.c +1 -0
+		/* Ensure pending TLS partial records are retried

*/

drivers/nvme/host/tcp.c +3 -0
+		if (nvme_tcp_queue_tls(queue))
+			queue->write_space(sk);
+

   queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue-

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

io_work);

   }
   read_unlock_bh(&sk->sk_callback_lock);

Minus the whitespace change:

Ah only just saw this, sent a V3 with the white-space fixed. Thanks for the review!

Regards,
Wilfred

Quoted 4 lines

Cheers,

Hannes