Skip to contentSkip to conversation

tls: Add opt-in zerocopy mode of sendfile()

PATCH Applied Maxim Mikityanskiy 3 participants 3 messages May 18, 2022 – May 19, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 tls: Add opt-in zerocopy mode of sendfile()

From: Boris Pismenny <[email protected]>

TLS device offload copies sendfile data to a bounce buffer before transmitting. It allows to maintain the valid MAC on TLS records when the file contents change and a part of TLS record has to be retransmitted on TCP level.

In many common use cases (like serving static files over HTTPS) the file contents are not changed on the fly. In many use cases breaking the connection is totally acceptable if the file is changed during transmission, because it would be received corrupted in any case.

This commit allows to optimize performance for such use cases to providing a new optional mode of TLS sendfile(), in which the extra copy is skipped. Removing this copy improves performance significantly, as TLS and TCP sendfile perform the same operations, and the only overhead is TLS header/trailer insertion.

The new mode can only be enabled with the new socket option named TLS_TX_ZEROCOPY_SENDFILE on per-socket basis. It preserves backwards compatibility with existing applications that rely on the copying behavior.

The new mode is safe, meaning that unsolicited modifications of the file being sent can't break integrity of the kernel. The worst thing that can happen is sending a corrupted TLS record, which is in any case not forbidden when using regular TCP sockets.

Sockets other than TLS device offload are not affected by the new socket option. The actual status of zerocopy sendfile can be queried with sock_diag.

Performance numbers in a single-core test with 24 HTTPS streams on nginx, under 100% CPU load:

* non-zerocopy: 33.6 Gbit/s
* zerocopy: 79.92 Gbit/s

CPU: Intel(R) Xeon(R) Platinum 8380 CPU @ 2.30GHz

---
 include/net/tls.h        |  1 +
 include/uapi/linux/tls.h |  2 ++
 net/tls/tls_device.c     | 53 ++++++++++++++++++++++++++++----------
 net/tls/tls_main.c       | 55 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 98 insertions(+), 13 deletions(-)
include/net/tls.h +1 -0
index b59f0a63292b..8017f1703447 100644
@@ -238,6 +238,7 @@ struct tls_context {
 
 	u8 tx_conf:3;
 	u8 rx_conf:3;
+	u8 zerocopy_sendfile:1;
 
 	int (*push_pending_record)(struct sock *sk, int flags);
 	void (*sk_write_space)(struct sock *sk);
include/uapi/linux/tls.h +2 -0
index 5f38be0ec0f3..ac39328eabe7 100644
@@ -39,6 +39,7 @@
 /* TLS socket options */
 #define TLS_TX			1	/* Set transmit parameters */
 #define TLS_RX			2	/* Set receive parameters */
+#define TLS_TX_ZEROCOPY_SENDFILE	3	/* transmit zerocopy sendfile */
 
 /* Supported versions */
 #define TLS_VERSION_MINOR(ver)	((ver) & 0xFF)
@@ -160,6 +161,7 @@ enum {
 	TLS_INFO_CIPHER,
 	TLS_INFO_TXCONF,
 	TLS_INFO_RXCONF,
+	TLS_INFO_ZC_SENDFILE,
 	__TLS_INFO_MAX,
 };
 #define TLS_INFO_MAX (__TLS_INFO_MAX - 1)
net/tls/tls_device.c +40 -13

105 lines not on this page · raw message

net/tls/tls_main.c +55 -0

91 lines not on this page · raw message

Signature · 1 line
2.25.1
maintainer Reviewed-by on v3 1/1
#1 lore raw
Quoted 10 lines · On Wed, 18 May 2022 12:27:31 +0300 Maxim Mikityanskiy wrote:

TLS device offload copies sendfile data to a bounce buffer before transmitting. It allows to maintain the valid MAC on TLS records when the file contents change and a part of TLS record has to be retransmitted on TCP level.

In many common use cases (like serving static files over HTTPS) the file contents are not changed on the fly. In many use cases breaking the connection is totally acceptable if the file is changed during transmission, because it would be received corrupted in any case.

Quoted 5 lines

This commit allows to optimize performance for such use cases to providing a new optional mode of TLS sendfile(), in which the extra copy is skipped. Removing this copy improves performance significantly, as TLS and TCP sendfile perform the same operations, and the only overhead is TLS header/trailer insertion.

#2 lore raw
In reply to Maxim Mikityanskiy
This patch was applied to netdev/net-next.git (master) by Paolo Abeni <[email protected]>:

Hello:

This patch was applied to netdev/net-next.git (master) by Paolo Abeni <[email protected]>:

Quoted 8 lines · On Wed, 18 May 2022 12:27:31 +0300 you wrote:

From: Boris Pismenny <[email protected]>

TLS device offload copies sendfile data to a bounce buffer before transmitting. It allows to maintain the valid MAC on TLS records when the file contents change and a part of TLS record has to be retransmitted on TCP level.

[...]

Here is the summary with links:
- [net-next,v3] tls: Add opt-in zerocopy mode of sendfile()

    https://git.kernel.org/netdev/net-next/c/c1318b39c7d3

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html