Skip to contentSkip to conversation

tls: Add opt-in zerocopy mode of sendfile()

PATCH Superseded Maxim Mikityanskiy 2 participants 4 messages May 11, 2022 – May 13, 2022

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 tls: Add opt-in zerocopy mode of sendfile()

From: Boris Pismenny <[email protected]>

TLS device offload copies sendfile data to a bounce buffer before transmitting. It allows to maintain the valid MAC on TLS records when the file contents change and a part of TLS record has to be retransmitted on TCP level.

In many common use cases (like serving static files over HTTPS) the file contents are not changed on the fly. In many use cases breaking the connection is totally acceptable if the file is changed during transmission, because it would be received corrupted in any case.

This commit allows to optimize performance for such use cases to providing a new optional mode of TLS sendfile(), in which the extra copy is skipped. Removing this copy improves performance significantly, as TLS and TCP sendfile perform the same operations, and the only overhead is TLS header/trailer insertion.

The new mode can only be enabled with the new socket option named TLS_TX_ZEROCOPY_SENDFILE on per-socket basis. It preserves backwards compatibility with existing applications that rely on the copying behavior.

The new mode is safe, meaning that unsolicited modifications of the file being sent can't break integrity of the kernel. The worst thing that can happen is sending a corrupted TLS record, which is in any case not forbidden when using regular TCP sockets.

Sockets other than TLS device offload are not affected by the new socket option.

Performance numbers in a single-core test with 12 HTTPS streams on nginx, under 100% CPU load:

* non-zerocopy: up to 23.5 Gbit/s
* zerocopy: up to 50.0 Gbit/s

---
 include/net/tls.h        |  1 +
 include/uapi/linux/tls.h |  1 +
 net/tls/tls_device.c     | 53 ++++++++++++++++++++++++++++++----------
 net/tls/tls_main.c       | 48 ++++++++++++++++++++++++++++++++++++
 4 files changed, 90 insertions(+), 13 deletions(-)
include/net/tls.h +1 -0
index b59f0a63292b..8017f1703447 100644
@@ -238,6 +238,7 @@ struct tls_context {
 
 	u8 tx_conf:3;
 	u8 rx_conf:3;
+	u8 zerocopy_sendfile:1;
 
 	int (*push_pending_record)(struct sock *sk, int flags);
 	void (*sk_write_space)(struct sock *sk);
include/uapi/linux/tls.h +1 -0
index 5f38be0ec0f3..47989b77ebcf 100644
@@ -39,6 +39,7 @@
 /* TLS socket options */
 #define TLS_TX			1	/* Set transmit parameters */
 #define TLS_RX			2	/* Set receive parameters */
+#define TLS_TX_ZEROCOPY_SENDFILE	3	/* transmit zerocopy sendfile */
 
 /* Supported versions */
 #define TLS_VERSION_MINOR(ver)	((ver) & 0xFF)
net/tls/tls_device.c +40 -13

105 lines not on this page · raw message

net/tls/tls_main.c +48 -0

72 lines not on this page · raw message

Signature · 1 line
2.25.1
maintainer
#1 lore raw
Quoted 21 lines · On Wed, 11 May 2022 15:15:25 +0300 Maxim Mikityanskiy wrote:

TLS device offload copies sendfile data to a bounce buffer before transmitting. It allows to maintain the valid MAC on TLS records when the file contents change and a part of TLS record has to be retransmitted on TCP level.

In many common use cases (like serving static files over HTTPS) the file contents are not changed on the fly. In many use cases breaking the connection is totally acceptable if the file is changed during transmission, because it would be received corrupted in any case.

This commit allows to optimize performance for such use cases to providing a new optional mode of TLS sendfile(), in which the extra copy is skipped. Removing this copy improves performance significantly, as TLS and TCP sendfile perform the same operations, and the only overhead is TLS header/trailer insertion.

The new mode can only be enabled with the new socket option named TLS_TX_ZEROCOPY_SENDFILE on per-socket basis. It preserves backwards compatibility with existing applications that rely on the copying behavior.

Quoted 7 lines

The new mode is safe, meaning that unsolicited modifications of the file being sent can't break integrity of the kernel. The worst thing that can happen is sending a corrupted TLS record, which is in any case not forbidden when using regular TCP sockets.

Sockets other than TLS device offload are not affected by the new socket option.

What about the reporting via sock diag? Am I misremembering something?

#2 lore raw
Quoted 27 lines · Jakub Kicinski
Quoted 26 lines · On Wed, 11 May 2022 15:15:25 +0300 Maxim Mikityanskiy wrote:

TLS device offload copies sendfile data to a bounce buffer before transmitting. It allows to maintain the valid MAC on TLS records when the file contents change and a part of TLS record has to be retransmitted on TCP level.

In many common use cases (like serving static files over HTTPS) the file contents are not changed on the fly. In many use cases breaking the connection is totally acceptable if the file is changed during transmission, because it would be received corrupted in any case.

This commit allows to optimize performance for such use cases to providing a new optional mode of TLS sendfile(), in which the extra copy is skipped. Removing this copy improves performance significantly, as TLS and TCP sendfile perform the same operations, and the only overhead is TLS header/trailer insertion.

The new mode can only be enabled with the new socket option named TLS_TX_ZEROCOPY_SENDFILE on per-socket basis. It preserves backwards compatibility with existing applications that rely on the copying behavior.

The new mode is safe, meaning that unsolicited modifications of the file being sent can't break integrity of the kernel. The worst thing that can happen is sending a corrupted TLS record, which is in any case not forbidden when using regular TCP sockets.

Quoted 4 lines
Quoted 2 lines

Sockets other than TLS device offload are not affected by the new socket option.

What about the reporting via sock diag? Am I misremembering something?

I recall we discussed that "zerocopy is active" can be restored as "hardware offload && setsockopt flag requested", and I saw that tls_get_info exposes the hardware offload state for the socket, so I thought the existing information in sock_diag was enough.

If you think, though, that it will be better to have an explicit indication of the zerocopy flag to sock_diag, I can add it, it's not a problem.

Does the rest of the patch look good to you?

Thanks,
Max

maintainer
#3 lore raw
Quoted 10 lines · On Fri, 13 May 2022 10:57:06 +0300 Maxim Mikityanskiy wrote:
Quoted 1 line

What about the reporting via sock diag? Am I misremembering something?

I recall we discussed that "zerocopy is active" can be restored as "hardware offload && setsockopt flag requested", and I saw that tls_get_info exposes the hardware offload state for the socket, so I thought the existing information in sock_diag was enough.

If you think, though, that it will be better to have an explicit indication of the zerocopy flag to sock_diag, I can add it, it's not a problem.

Yeah, my thinking is that if a user reports problems to me I'd like to be able to log in to the box and see what they have configured on the socket. ss is quick and easy to type. I'm not aware of tooling that'd grab descriptors for sockets and do getsockopt() (even tho technically it is doable), are there any? I'm not feeling strong about sock_diag specifically being the way but for such a potentially correctness- -impacting optimization we should give admins an easy way to interrogate if it's enabled or not.

Quoted 1 line

Does the rest of the patch look good to you?

Yup!