tls: Add opt-in zerocopy mode of sendfile()
Subsystems (3)
PATCH v2 1/1 tls: Add opt-in zerocopy mode of sendfile()
From: Boris Pismenny <[email protected]>
TLS device offload copies sendfile data to a bounce buffer before transmitting. It allows to maintain the valid MAC on TLS records when the file contents change and a part of TLS record has to be retransmitted on TCP level.
In many common use cases (like serving static files over HTTPS) the file contents are not changed on the fly. In many use cases breaking the connection is totally acceptable if the file is changed during transmission, because it would be received corrupted in any case.
This commit allows to optimize performance for such use cases to providing a new optional mode of TLS sendfile(), in which the extra copy is skipped. Removing this copy improves performance significantly, as TLS and TCP sendfile perform the same operations, and the only overhead is TLS header/trailer insertion.
The new mode can only be enabled with the new socket option named TLS_TX_ZEROCOPY_SENDFILE on per-socket basis. It preserves backwards compatibility with existing applications that rely on the copying behavior.
The new mode is safe, meaning that unsolicited modifications of the file being sent can't break integrity of the kernel. The worst thing that can happen is sending a corrupted TLS record, which is in any case not forbidden when using regular TCP sockets.
Sockets other than TLS device offload are not affected by the new socket option.
Performance numbers in a single-core test with 12 HTTPS streams on nginx, under 100% CPU load:
* non-zerocopy: up to 23.5 Gbit/s
* zerocopy: up to 50.0 Gbit/s
--- include/net/tls.h | 1 + include/uapi/linux/tls.h | 1 + net/tls/tls_device.c | 53 ++++++++++++++++++++++++++++++---------- net/tls/tls_main.c | 48 ++++++++++++++++++++++++++++++++++++ 4 files changed, 90 insertions(+), 13 deletions(-)
include/net/tls.h +1 -0
@@ -238,6 +238,7 @@ struct tls_context { u8 tx_conf:3; u8 rx_conf:3; + u8 zerocopy_sendfile:1;int (*push_pending_record)(struct sock *sk,int flags);void (*sk_write_space)(struct sock *sk);
include/uapi/linux/tls.h +1 -0
@@ -39,6 +39,7 @@/* TLS socket options */ #define TLS_TX 1/* Set transmit parameters */ #define TLS_RX 2/* Set receive parameters */ +#define TLS_TX_ZEROCOPY_SENDFILE 3/* transmit zerocopy sendfile */ /* Supported versions */ #define TLS_VERSION_MINOR (ver) ((ver) & 0xFF)
net/tls/tls_device.c +40 -13
105 lines not on this page · raw message
net/tls/tls_main.c +48 -0
72 lines not on this page · raw message
Signature · 1 line
2.25.1
Jakub Kicinski
maintainerQuoted 21 lines · On Wed, 11 May 2022 15:15:25 +0300 Maxim Mikityanskiy wrote:
TLS device offload copies sendfile data to a bounce buffer before transmitting. It allows to maintain the valid MAC on TLS records when the file contents change and a part of TLS record has to be retransmitted on TCP level.
In many common use cases (like serving static files over HTTPS) the file contents are not changed on the fly. In many use cases breaking the connection is totally acceptable if the file is changed during transmission, because it would be received corrupted in any case.
This commit allows to optimize performance for such use cases to providing a new optional mode of TLS
sendfile(), in which the extra copy is skipped. Removing this copy improves performance significantly, as TLS and TCP sendfile perform the same operations, and the only overhead is TLS header/trailer insertion.The new mode can only be enabled with the new socket option named
TLS_TX_ZEROCOPY_SENDFILEon per-socket basis. It preserves backwards compatibility with existing applications that rely on the copying behavior.
Quoted 7 lines
The new mode is safe, meaning that unsolicited modifications of the file being sent can't break integrity of the kernel. The worst thing that can happen is sending a corrupted TLS record, which is in any case not forbidden when using regular TCP sockets.
Sockets other than TLS device offload are not affected by the new socket option.
What about the reporting via sock diag? Am I misremembering something?
Quoted 27 lines · Jakub Kicinski
Quoted 26 lines · On Wed, 11 May 2022 15:15:25 +0300 Maxim Mikityanskiy wrote:
TLS device offload copies sendfile data to a bounce buffer before transmitting. It allows to maintain the valid MAC on TLS records when the file contents change and a part of TLS record has to be retransmitted on TCP level.
In many common use cases (like serving static files over HTTPS) the file contents are not changed on the fly. In many use cases breaking the connection is totally acceptable if the file is changed during transmission, because it would be received corrupted in any case.
This commit allows to optimize performance for such use cases to providing a new optional mode of TLS
sendfile(), in which the extra copy is skipped. Removing this copy improves performance significantly, as TLS and TCP sendfile perform the same operations, and the only overhead is TLS header/trailer insertion.The new mode can only be enabled with the new socket option named
TLS_TX_ZEROCOPY_SENDFILEon per-socket basis. It preserves backwards compatibility with existing applications that rely on the copying behavior.The new mode is safe, meaning that unsolicited modifications of the file being sent can't break integrity of the kernel. The worst thing that can happen is sending a corrupted TLS record, which is in any case not forbidden when using regular TCP sockets.
Quoted 4 lines
Quoted 2 lines
Sockets other than TLS device offload are not affected by the new socket option.
What about the reporting via sock diag? Am I misremembering something?
I recall we discussed that "zerocopy is active" can be restored as "hardware offload && setsockopt flag requested", and I saw that tls_get_info exposes the hardware offload state for the socket, so I thought the existing information in sock_diag was enough.
If you think, though, that it will be better to have an explicit indication of the zerocopy flag to sock_diag, I can add it, it's not a problem.
Does the rest of the patch look good to you?
Thanks,
Max
Jakub Kicinski
maintainerQuoted 10 lines · On Fri, 13 May 2022 10:57:06 +0300 Maxim Mikityanskiy wrote:
Quoted 1 line
What about the reporting via sock diag? Am I misremembering something?
I recall we discussed that "zerocopy is active" can be restored as "hardware offload && setsockopt flag requested", and I saw that
tls_get_infoexposes the hardware offload state for the socket, so I thought the existing information insock_diagwas enough.If you think, though, that it will be better to have an explicit indication of the zerocopy flag to
sock_diag, I can add it, it's not a problem.
Yeah, my thinking is that if a user reports problems to me I'd like to be able to log in to the box and see what they have configured on the socket. ss is quick and easy to type. I'm not aware of tooling that'd grab descriptors for sockets and do getsockopt() (even tho technically it is doable), are there any? I'm not feeling strong about sock_diag specifically being the way but for such a potentially correctness- -impacting optimization we should give admins an easy way to interrogate if it's enabled or not.
Quoted 1 line
Does the rest of the patch look good to you?
Yup!