nvme/tcp: handle tls partially sent records in write_space()
PATCH v2 1/1 nvme/tcp: handle tls partially sent records in write_space()
From: Wilfred Mallawa <[email protected]>
With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for example, hitting an EAGAIN from tcp_sendmsg_locked(). This is not known to the NVMe TCP driver, as the TLS layer successfully generated a record.
Typically, the TLS write_space() callback would ensure such records are retried, but in the NVMe TCP Host driver, write_space() invokes nvme_tcp_write_space(). This causes a partially sent record in the TLS TX list to timeout after not being retried.
This patch fixes the above by calling queue->write_space(), which calls into the TLS layer to retry any pending records.
---
V2:
- Unconditionally invoke TLS write_space(). This means we don't need to export tls_is_partially_sent_record()
---drivers/nvme/host/tcp.c | 5 +++++
1 file changed, 5 insertions(+)
drivers/nvme/host/tcp.c +5 -0
@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock *sk)read_lock_bh (&sk->sk_callback_lock); queue = sk->sk_user_data; +if (likely (queue &&sk_stream_is_writeable (sk))) {clear_bit (SOCK_NOSPACE , &sk->sk_socket->flags); +/* Ensure pending TLS partial records are retried */ +if (nvme_tcp_queue_tls (queue)) + queue->write_space (sk); +queue_work_on (queue->io_cpu, nvme_tcp_wq, &queue->io_work); }read_unlock_bh (&sk->sk_callback_lock);
Signature · 1 line
2.51.0
Quoted 30 lines · Wilfred Mallawa
From: Wilfred Mallawa <[email protected]>
With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for example, hitting an EAGAIN from
tcp_sendmsg_locked(). This is not known to the NVMe TCP driver, as the TLS layer successfully generated a record.Typically, the TLS
write_space()callback would ensure such records are retried, but in the NVMe TCP Host driver,write_space()invokesnvme_tcp_write_space(). This causes a partially sent record in the TLS TX list to timeout after not being retried.This patch fixes the above by calling
queue->write_space(), which calls into the TLS layer to retry any pending records.Fixes be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")Signed-off-by Wilfred Mallawa <[email protected]>---
V2:
- Unconditionally invoke TLSwrite_space(). This means we don't need to exporttls_is_partially_sent_record()
---drivers/nvme/host/tcp.c| 5 +++++1 file changed, 5 insertions(+)drivers/nvme/host/tcp.c +0
-0
drivers/nvme/host/tcp.c static void nvme_tcp_write_space(struct sock *sk) · line 1082
Quoted 4 lines
drivers/nvme/host/tcp.c +1
-0@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock *sk)read_lock_bh (&sk->sk_callback_lock); queue = sk->sk_user_data; +
White space change... Remove please.
drivers/nvme/host/tcp.c static void nvme_tcp_write_space(struct sock *sk) · line 1088
Quoted 6 lines
drivers/nvme/host/tcp.c +4
-0if (likely (queue &&sk_stream_is_writeable (sk))) {clear_bit (SOCK_NOSPACE , &sk->sk_socket->flags); +/* Ensure pending TLS partial records are retried */ +if (nvme_tcp_queue_tls (queue)) + queue->write_space (sk); +
Maybe remove this empty line too.
Quoted 3 lines
drivers/nvme/host/tcp.c +0
-0queue_work_on (queue->io_cpu, nvme_tcp_wq, &queue->io_work); }read_unlock_bh (&sk->sk_callback_lock);
Signature · 2 lines
Damien Le Moal Western Digital Research
Quoted 33 lines · Wilfred Mallawa
From: Wilfred Mallawa <[email protected]>
With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for example, hitting an EAGAIN from
tcp_sendmsg_locked(). This is not known to the NVMe TCP driver, as the TLS layer successfully generated a record.Typically, the TLS
write_space()callback would ensure such records are retried, but in the NVMe TCP Host driver,write_space()invokesnvme_tcp_write_space(). This causes a partially sent record in the TLS TX list to timeout after not being retried.This patch fixes the above by calling
queue->write_space(), which calls into the TLS layer to retry any pending records.Fixes be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")Signed-off-by Wilfred Mallawa <[email protected]>---
V2:
- Unconditionally invoke TLSwrite_space(). This means we don't need to exporttls_is_partially_sent_record()
---drivers/nvme/host/tcp.c| 5 +++++1 file changed, 5 insertions(+)drivers/nvme/host/tcp.c +0
-0@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock *sk)read_lock_bh (&sk->sk_callback_lock); queue = sk->sk_user_data;
drivers/nvme/host/tcp.c static void nvme_tcp_write_space(struct sock *sk) · line 1091
Quoted 10 lines
drivers/nvme/host/tcp.c +5
-0+if (likely (queue &&sk_stream_is_writeable (sk))) {clear_bit (SOCK_NOSPACE , &sk->sk_socket->flags); +/* Ensure pending TLS partial records are retried */ +if (nvme_tcp_queue_tls (queue)) + queue->write_space (sk); +queue_work_on (queue->io_cpu, nvme_tcp_wq, &queue->io_work); }read_unlock_bh (&sk->sk_callback_lock);
Minus the whitespace change:
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
Quoted 46 lines · Damien Le Moal
Quoted 38 lines · Wilfred Mallawa
From: Wilfred Mallawa <[email protected]>
With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for
example, hitting an EAGAIN fromtcp_sendmsg_locked(). This is not known
to the NVMe TCP driver, as the TLS layer successfully generated a record.Typically, the TLS
write_space()callback would ensure such records are
retried, but in the NVMe TCP Host driver,write_space()invokesnvme_tcp_write_space(). This causes a partially sent record in the TLS TX
list to timeout after not being retried.This patch fixes the above by calling
queue->write_space(), which calls
into the TLS layer to retry any pending records.Fixes be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")Signed-off-by Wilfred Mallawa <[email protected]>---
V2:
- Unconditionally invoke TLSwrite_space(). This means we don't need
to exporttls_is_partially_sent_record()
---drivers/nvme/host/tcp.c| 5 +++++1 file changed, 5 insertions(+)drivers/nvme/host/tcp.c +0
-0@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock *sk)Quoted 4 lines
read_lock_bh(&sk->sk_callback_lock);
queue =sk->sk_user_data;
+White space change... Remove please.
Quoted 9 lines
drivers/nvme/host/tcp.c static void nvme_tcp_write_space(struct sock *sk) · line 1088
Quoted 7 lines
if (
likely(queue && sk_stream_is_writeable(sk))) {clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);drivers/nvme/host/tcp.c +1
-0+/* Ensure pending TLS partial records are retried */
drivers/nvme/host/tcp.c +3
-0+if (nvme_tcp_queue_tls (queue)) + queue->write_space (sk); +Maybe remove this empty line too.
Okay, Fixed in V3. Thanks!
Wilfred
Quoted 6 lines
Quoted 5 lines
queue_work_on(queue->io_cpu,
nvme_tcp_wq, &queue-Quoted 1 line
io_work);}
read_unlock_bh(&sk->sk_callback_lock);
Quoted 51 lines · Hannes Reinecke
Quoted 50 lines · Wilfred Mallawa
From: Wilfred Mallawa <[email protected]>
With TLS enabled, records that are encrypted and appended to TLS TX list can fail to see a retry if the underlying TCP socket is busy, for
example, hitting an EAGAIN fromtcp_sendmsg_locked(). This is not known
to the NVMe TCP driver, as the TLS layer successfully generated a record.Typically, the TLS
write_space()callback would ensure such records are
retried, but in the NVMe TCP Host driver,write_space()invokesnvme_tcp_write_space(). This causes a partially sent record in the TLS TX
list to timeout after not being retried.This patch fixes the above by calling
queue->write_space(), which calls
into the TLS layer to retry any pending records.Fixes be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")Signed-off-by Wilfred Mallawa <[email protected]>---
V2:
- Unconditionally invoke TLSwrite_space(). This means we don't need
to exporttls_is_partially_sent_record()
---drivers/nvme/host/tcp.c| 5 +++++1 file changed, 5 insertions(+)drivers/nvme/host/tcp.c +0
-0@@ -1079,8 +1079,13 @@ static void nvme_tcp_write_space(struct sock *sk)
read_lock_bh(&sk->sk_callback_lock);
queue =sk->sk_user_data;
+
if (likely(queue && sk_stream_is_writeable(sk))) {clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);drivers/nvme/host/tcp.c +1
-0+/* Ensure pending TLS partial records are retried */
drivers/nvme/host/tcp.c +3
-0+if (nvme_tcp_queue_tls (queue)) + queue->write_space (sk); +queue_work_on(queue->io_cpu,
nvme_tcp_wq, &queue-
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
io_work);}
read_unlock_bh(&sk->sk_callback_lock);Minus the whitespace change:
Reviewed-by Hannes Reinecke <[email protected]>
Ah only just saw this, sent a V3 with the white-space fixed. Thanks for the review!
Regards,
Wilfred
Quoted 4 lines
Cheers,
Hannes