Skip to contentSkip to conversation

tls: implement key updates for TLS1.3

PATCH Superseded Sabrina Dubroca 8 participants 32 messages Jan 17, 2023 – Jan 25, 2023

Subsystems (4)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Sabrina Dubroca

author maintainer
#0 lore raw

cover tls: implement key updates for TLS1.3

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace
3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Note: in a future series, I'll clean up tls_set_sw_offload and eliminate the per-cipher copy-paste using tls_cipher_size_desc.

[1] https://www.rfc-editor.org/rfc/rfc8446#section-4.6.3
[2] https://gitlab.com/gnutls/gnutls/-/merge_requests/1625

Sabrina Dubroca (5):
tls: remove tls_context argument from tls_set_sw_offload
tls: block decryption when a rekey is pending
tls: implement rekey for TLS1.3
selftests: tls: add key_generation argument to tls_crypto_info_init
selftests: tls: add rekey tests

 include/net/tls.h                 |   4 +
 net/tls/tls.h                     |   3 +-
 net/tls/tls_device.c              |   2 +-
 net/tls/tls_main.c                |  32 +++-
 net/tls/tls_sw.c                  | 169 +++++++++++++++----
 tools/testing/selftests/net/tls.c | 272 +++++++++++++++++++++++++++++-
 6 files changed, 434 insertions(+), 48 deletions(-)
Signature · 1 line
2.38.1
author maintainer
#3 lore raw

PATCH 2/5 tls: block decryption when a rekey is pending

When a TLS handshake record carrying a KeyUpdate message is received, all subsequent records will be encrypted with a new key. We need to stop decrypting incoming records with the old key, and wait until userspace provides a new key.

Make a note of this in the RX context just after decrypting that record, and stop recvmsg/splice calls with EKEYEXPIRED until the new key is available.

---
 include/net/tls.h |  4 ++++
 net/tls/tls_sw.c  | 44 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)
include/net/tls.h +4 -0
index 154949c7b0c8..297732f23804 100644
@@ -69,8 +69,11 @@ extern const struct tls_cipher_size_desc tls_cipher_size_desc[];
 
 #define TLS_CRYPTO_INFO_READY(info)	((info)->cipher_type)
 
+#define TLS_RECORD_TYPE_HANDSHAKE	0x16
 #define TLS_RECORD_TYPE_DATA		0x17
 
+#define TLS_HANDSHAKE_KEYUPDATE		24	/* rfc8446 B.3: Key update */
+
 #define TLS_AAD_SPACE_SIZE		13
 
 #define MAX_IV_SIZE			16
@@ -145,6 +148,7 @@ struct tls_sw_context_rx {
 
 	struct tls_strparser strp;
 
+	bool key_update_pending;
 	atomic_t decrypt_pending;
 	/* protect crypto_wait with decrypt_pending*/
 	spinlock_t decrypt_compl_lock;
net/tls/tls_sw.c +44 -0

74 lines not on this page · raw message

Signature · 1 line
2.38.1
author maintainer
#1 lore raw

PATCH 1/5 tls: remove tls_context argument from tls_set_sw_offload

In reply to Sabrina Dubroca

It's not really needed since we end up refetching it as tls_ctx. We can also remove the NULL check, since we have already dereferenced ctx in do_tls_setsockopt_conf.

---
 net/tls/tls.h        |  2 +-
 net/tls/tls_device.c |  2 +-
 net/tls/tls_main.c   |  4 ++--
 net/tls/tls_sw.c     | 11 +++--------
 4 files changed, 7 insertions(+), 12 deletions(-)
net/tls/tls.h +1 -1
index 0e840a0c3437..34d0fe814600 100644
@@ -90,7 +90,7 @@ int tls_sk_attach(struct sock *sk, int optname, char __user *optval,
 		  unsigned int optlen);
 void tls_err_abort(struct sock *sk, int err);
 
-int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx);
+int tls_set_sw_offload(struct sock *sk, int tx);
 void tls_update_rx_zc_capable(struct tls_context *tls_ctx);
 void tls_sw_strparser_arm(struct sock *sk, struct tls_context *ctx);
 void tls_sw_strparser_done(struct tls_context *tls_ctx);
net/tls/tls_device.c +1 -1
index 6c593788dc25..c149f36b42ee 100644
@@ -1291,7 +1291,7 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
 	context->resync_nh_reset = 1;
 
 	ctx->priv_ctx_rx = context;
-	rc = tls_set_sw_offload(sk, ctx, 0);
+	rc = tls_set_sw_offload(sk, 0);
 	if (rc)
 		goto release_ctx;
 
net/tls/tls_main.c +2 -2
index 3735cb00905d..fb1da1780f50 100644
@@ -772,7 +772,7 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXDEVICE);
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXDEVICE);
 		} else {
-			rc = tls_set_sw_offload(sk, ctx, 1);
+			rc = tls_set_sw_offload(sk, 1);
 			if (rc)
 				goto err_crypto_info;
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXSW);
@@ -786,7 +786,7 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXDEVICE);
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXDEVICE);
 		} else {
-			rc = tls_set_sw_offload(sk, ctx, 0);
+			rc = tls_set_sw_offload(sk, 0);
 			if (rc)
 				goto err_crypto_info;
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXSW);
net/tls/tls_sw.c +3 -8
index 9ed978634125..238562f9081b 100644
@@ -2462,10 +2462,10 @@ void tls_update_rx_zc_capable(struct tls_context *tls_ctx)
 		tls_ctx->prot_info.version != TLS_1_3_VERSION;
 }
 
-int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
+int tls_set_sw_offload(struct sock *sk, int tx)
 {
-	struct tls_context *tls_ctx = tls_get_ctx(sk);
-	struct tls_prot_info *prot = &tls_ctx->prot_info;
+	struct tls_context *ctx = tls_get_ctx(sk);
+	struct tls_prot_info *prot = &ctx->prot_info;
 	struct tls_crypto_info *crypto_info;
 	struct tls_sw_context_tx *sw_ctx_tx = NULL;
 	struct tls_sw_context_rx *sw_ctx_rx = NULL;
@@ -2477,11 +2477,6 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
 	size_t keysize;
 	int rc = 0;
 
-	if (!ctx) {
-		rc = -EINVAL;
-		goto out;
-	}
-
 	if (tx) {
 		if (!ctx->priv_ctx_tx) {
 			sw_ctx_tx = kzalloc(sizeof(*sw_ctx_tx), GFP_KERNEL);
Signature · 1 line
2.38.1
author maintainer
#13 lore raw

PATCH 5/5 selftests: tls: add rekey tests

In reply to Sabrina Dubroca
---
 tools/testing/selftests/net/tls.c | 258 ++++++++++++++++++++++++++++++
 1 file changed, 258 insertions(+)
tools/testing/selftests/net/tls.c +258 -0
index 5f3adb28fee1..97c20e2246e1 100644
@@ -1453,6 +1453,264 @@ TEST_F(tls, shutdown_reuse)
 	EXPECT_EQ(errno, EISCONN);
 }
 
+#define TLS_RECORD_TYPE_HANDSHAKE      0x16
+/* key_update, length 1, update_not_requested */
+static const char key_update_msg[] = "\x18\x00\x00\x01\x00";
+static void tls_send_keyupdate(struct __test_metadata *_metadata, int fd)
+{
+	size_t len = sizeof(key_update_msg);
+
+	EXPECT_EQ(tls_send_cmsg(fd, TLS_RECORD_TYPE_HANDSHAKE,
+				(char *)key_update_msg, len, 0),
+		  len);
+}
+
+static void tls_recv_keyupdate(struct __test_metadata *_metadata, int fd, int flags)
+{
+	char buf[100];
+
+	EXPECT_EQ(tls_recv_cmsg(_metadata, fd, TLS_RECORD_TYPE_HANDSHAKE, buf, sizeof(buf), flags),
+		  sizeof(key_update_msg));
+	EXPECT_EQ(memcmp(buf, key_update_msg, sizeof(key_update_msg)), 0);
+}
+
+/* set the key to 0 then 1 for RX, immediately to 1 for TX */
+TEST_F(tls_basic, rekey_rx)
+{
+	struct tls_crypto_info_keys tls12_0, tls12_1;
+	char const *test_str = "test_message";
+	int send_len = strlen(test_str) + 1;
+	char buf[20];
+	int ret;
+
+	if (self->notls)
+		return;
+
+	tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128,
+			     &tls12_0, 0);
+	tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128,
+			     &tls12_1, 1);
+
+
+	ret = setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_1, tls12_1.len);
+	ASSERT_EQ(ret, 0);
+
+	ret = setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_0, tls12_0.len);
+	ASSERT_EQ(ret, 0);
+
+	ret = setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_1, tls12_1.len);
+	EXPECT_EQ(ret, 0);
+
+	EXPECT_EQ(send(self->fd, test_str, send_len, 0), send_len);
+	EXPECT_EQ(recv(self->cfd, buf, send_len, 0), send_len);
+	EXPECT_EQ(memcmp(buf, test_str, send_len), 0);
+}
+
+/* set the key to 0 then 1 for TX, immediately to 1 for RX */
+TEST_F(tls_basic, rekey_tx)
+{
+	struct tls_crypto_info_keys tls12_0, tls12_1;
+	char const *test_str = "test_message";
+	int send_len = strlen(test_str) + 1;
+	char buf[20];
+	int ret;
+
+	if (self->notls)
+		return;
+
+	tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128,
+			     &tls12_0, 0);
+	tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128,
+			     &tls12_1, 1);
+
+
+	ret = setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_0, tls12_0.len);
+	ASSERT_EQ(ret, 0);
+
+	ret = setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_1, tls12_1.len);
+	ASSERT_EQ(ret, 0);
+
+	ret = setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_1, tls12_1.len);
+	EXPECT_EQ(ret, 0);
+
+	EXPECT_EQ(send(self->fd, test_str, send_len, 0), send_len);
+	EXPECT_EQ(recv(self->cfd, buf, send_len, 0), send_len);
+	EXPECT_EQ(memcmp(buf, test_str, send_len), 0);
+}
+
+TEST_F(tls, rekey)
+{
+	char const *test_str_1 = "test_message_before_rekey";
+	char const *test_str_2 = "test_message_after_rekey";
+	struct tls_crypto_info_keys tls12;
+	int send_len;
+	char buf[100];
+
+	if (variant->tls_version != TLS_1_3_VERSION)
+		return;
+
+	/* initial send/recv */
+	send_len = strlen(test_str_1) + 1;
+	EXPECT_EQ(send(self->fd, test_str_1, send_len, 0), send_len);
+	EXPECT_EQ(recv(self->cfd, buf, send_len, 0), send_len);
+	EXPECT_EQ(memcmp(buf, test_str_1, send_len), 0);
+
+	/* update TX key */
+	tls_send_keyupdate(_metadata, self->fd);
+	tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 1);
+	EXPECT_EQ(setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12, tls12.len), 0);
+
+	/* send after rekey */
+	send_len = strlen(test_str_2) + 1;
+	EXPECT_EQ(send(self->fd, test_str_2, send_len, 0), send_len);
+
+	/* can't receive the KeyUpdate without a control message */
+	EXPECT_EQ(recv(self->cfd, buf, send_len, 0), -1);
+
+	/* get KeyUpdate */
+	tls_recv_keyupdate(_metadata, self->cfd, 0);
+
+	/* recv blocking -> -EKEYEXPIRED */
+	EXPECT_EQ(recv(self->cfd, buf, sizeof(buf), 0), -1);
+	EXPECT_EQ(errno, EKEYEXPIRED);
+
+	/* recv non-blocking -> -EKEYEXPIRED */
+	EXPECT_EQ(recv(self->cfd, buf, sizeof(buf), MSG_DONTWAIT), -1);
+	EXPECT_EQ(errno, EKEYEXPIRED);
+
+	/* update RX key */
+	EXPECT_EQ(setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12, tls12.len), 0);
+
+	/* recv after rekey */
+	EXPECT_NE(recv(self->cfd, buf, send_len, 0), -1);
+	EXPECT_EQ(memcmp(buf, test_str_2, send_len), 0);
+}
+
+TEST_F(tls, rekey_peek)
+{
+	char const *test_str_1 = "test_message_before_rekey";
+	struct tls_crypto_info_keys tls12;
+	int send_len;
+	char buf[100];
+
+	if (variant->tls_version != TLS_1_3_VERSION)
+		return;
+
+	send_len = strlen(test_str_1) + 1;
+	EXPECT_EQ(send(self->fd, test_str_1, send_len, 0), send_len);
+
+	/* update TX key */
+	tls_send_keyupdate(_metadata, self->fd);
+	tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 1);
+	EXPECT_EQ(setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12, tls12.len), 0);
+
+	EXPECT_EQ(recv(self->cfd, buf, sizeof(buf), MSG_PEEK), send_len);
+	EXPECT_EQ(memcmp(buf, test_str_1, send_len), 0);
+
+	EXPECT_EQ(recv(self->cfd, buf, send_len, 0), send_len);
+	EXPECT_EQ(memcmp(buf, test_str_1, send_len), 0);
+
+	/* can't receive the KeyUpdate without a control message */
+	EXPECT_EQ(recv(self->cfd, buf, send_len, MSG_PEEK), -1);
+
+	/* peek KeyUpdate */
+	tls_recv_keyupdate(_metadata, self->cfd, MSG_PEEK);
+
+	/* get KeyUpdate */
+	tls_recv_keyupdate(_metadata, self->cfd, 0);
+
+	/* update RX key */
+	EXPECT_EQ(setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12, tls12.len), 0);
+}
+
+TEST_F(tls, splice_rekey)
+{
+	int send_len = TLS_PAYLOAD_MAX_LEN / 2;
+	char mem_send[TLS_PAYLOAD_MAX_LEN];
+	char mem_recv[TLS_PAYLOAD_MAX_LEN];
+	struct tls_crypto_info_keys tls12;
+	int p[2];
+
+	if (variant->tls_version != TLS_1_3_VERSION)
+		return;
+
+	memrnd(mem_send, sizeof(mem_send));
+
+	ASSERT_GE(pipe(p), 0);
+	EXPECT_EQ(send(self->fd, mem_send, send_len, 0), send_len);
+
+	/* update TX key */
+	tls_send_keyupdate(_metadata, self->fd);
+	tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 1);
+	EXPECT_EQ(setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12, tls12.len), 0);
+
+	EXPECT_EQ(send(self->fd, mem_send, send_len, 0), send_len);
+
+	EXPECT_EQ(splice(self->cfd, NULL, p[1], NULL, TLS_PAYLOAD_MAX_LEN, 0), send_len);
+	EXPECT_EQ(read(p[0], mem_recv, send_len), send_len);
+	EXPECT_EQ(memcmp(mem_send, mem_recv, send_len), 0);
+
+	/* can't splice the KeyUpdate */
+	EXPECT_EQ(splice(self->cfd, NULL, p[1], NULL, TLS_PAYLOAD_MAX_LEN, 0), -1);
+	EXPECT_EQ(errno, EINVAL);
+
+	/* peek KeyUpdate */
+	tls_recv_keyupdate(_metadata, self->cfd, MSG_PEEK);
+
+	/* get KeyUpdate */
+	tls_recv_keyupdate(_metadata, self->cfd, 0);
+
+	/* can't splice before updating the key */
+	EXPECT_EQ(splice(self->cfd, NULL, p[1], NULL, TLS_PAYLOAD_MAX_LEN, 0), -1);
+	EXPECT_EQ(errno, EKEYEXPIRED);
+
+	/* update RX key */
+	EXPECT_EQ(setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12, tls12.len), 0);
+
+	EXPECT_EQ(splice(self->cfd, NULL, p[1], NULL, TLS_PAYLOAD_MAX_LEN, 0), send_len);
+	EXPECT_EQ(read(p[0], mem_recv, send_len), send_len);
+	EXPECT_EQ(memcmp(mem_send, mem_recv, send_len), 0);
+}
+
+TEST_F(tls, rekey_getsockopt)
+{
+	struct tls_crypto_info_keys tls12;
+	struct tls_crypto_info_keys tls12_get;
+	socklen_t len;
+
+	tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 0);
+
+	len = tls12.len;
+	EXPECT_EQ(getsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_get, &len), 0);
+	EXPECT_EQ(len, tls12.len);
+	EXPECT_EQ(memcmp(&tls12_get, &tls12, tls12.len), 0);
+
+	len = tls12.len;
+	EXPECT_EQ(getsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_get, &len), 0);
+	EXPECT_EQ(len, tls12.len);
+	EXPECT_EQ(memcmp(&tls12_get, &tls12, tls12.len), 0);
+
+	if (variant->tls_version != TLS_1_3_VERSION)
+		return;
+
+	tls_send_keyupdate(_metadata, self->fd);
+	tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 1);
+	EXPECT_EQ(setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12, tls12.len), 0);
+
+	tls_recv_keyupdate(_metadata, self->cfd, 0);
+	EXPECT_EQ(setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12, tls12.len), 0);
+
+	len = tls12.len;
+	EXPECT_EQ(getsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_get, &len), 0);
+	EXPECT_EQ(len, tls12.len);
+	EXPECT_EQ(memcmp(&tls12_get, &tls12, tls12.len), 0);
+
+	len = tls12.len;
+	EXPECT_EQ(getsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_get, &len), 0);
+	EXPECT_EQ(len, tls12.len);
+	EXPECT_EQ(memcmp(&tls12_get, &tls12, tls12.len), 0);
+}
+
 FIXTURE(tls_err)
 {
 	int fd, cfd;
Signature · 1 line
2.38.1
author maintainer
#12 lore raw

PATCH 4/5 selftests: tls: add key_generation argument to tls_crypto_info_init

In reply to Sabrina Dubroca

This allows us to generate different keys, so that we can test that rekey is using the correct one.

---
 tools/testing/selftests/net/tls.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
tools/testing/selftests/net/tls.c +8 -6
index 2cbb12736596..5f3adb28fee1 100644
@@ -38,9 +38,11 @@ struct tls_crypto_info_keys {
 };
 
 static void tls_crypto_info_init(uint16_t tls_version, uint16_t cipher_type,
-				 struct tls_crypto_info_keys *tls12)
+				 struct tls_crypto_info_keys *tls12,
+				 char key_generation)
 {
-	memset(tls12, 0, sizeof(*tls12));
+	memset(tls12, key_generation, sizeof(*tls12));
+	memset(tls12, 0, sizeof(struct tls_crypto_info));
 
 	switch (cipher_type) {
 	case TLS_CIPHER_CHACHA20_POLY1305:
@@ -312,7 +314,7 @@ FIXTURE_SETUP(tls)
 	int ret;
 
 	tls_crypto_info_init(variant->tls_version, variant->cipher_type,
-			     &tls12);
+			     &tls12, 0);
 
 	ulp_sock_pair(_metadata, &self->fd, &self->cfd, &self->notls);
 
@@ -1071,7 +1073,7 @@ TEST_F(tls, bidir)
 		struct tls_crypto_info_keys tls12;
 
 		tls_crypto_info_init(variant->tls_version, variant->cipher_type,
-				     &tls12);
+				     &tls12, 0);
 
 		ret = setsockopt(self->fd, SOL_TLS, TLS_RX, &tls12,
 				 tls12.len);
@@ -1479,7 +1481,7 @@ FIXTURE_SETUP(tls_err)
 	int ret;
 
 	tls_crypto_info_init(variant->tls_version, TLS_CIPHER_AES_GCM_128,
-			     &tls12);
+			     &tls12, 0);
 
 	ulp_sock_pair(_metadata, &self->fd, &self->cfd, &self->notls);
 	ulp_sock_pair(_metadata, &self->fd2, &self->cfd2, &self->notls);
@@ -1771,7 +1773,7 @@ TEST(tls_v6ops) {
 	int sfd, ret, fd;
 	socklen_t len, len2;
 
-	tls_crypto_info_init(TLS_1_2_VERSION, TLS_CIPHER_AES_GCM_128, &tls12);
+	tls_crypto_info_init(TLS_1_2_VERSION, TLS_CIPHER_AES_GCM_128, &tls12, 0);
 
 	addr.sin6_family = AF_INET6;
 	addr.sin6_addr = in6addr_any;
Signature · 1 line
2.38.1
author maintainer
#5 lore raw

PATCH 3/5 tls: implement rekey for TLS1.3

In reply to Sabrina Dubroca

This adds the possibility to change the key and IV when using TLS1.3. Changing the cipher or TLS version is not supported.

Once we have updated the RX key, we can unblock the receive side.

This change only affects tls_sw, since 1.3 offload isn't supported.

---
 net/tls/tls.h        |   3 +-
 net/tls/tls_device.c |   2 +-
 net/tls/tls_main.c   |  32 ++++++++++--
 net/tls/tls_sw.c     | 120 ++++++++++++++++++++++++++++++++-----------
 4 files changed, 120 insertions(+), 37 deletions(-)
net/tls/tls.h +2 -1
index 34d0fe814600..6f9c85eaa9c5 100644
@@ -90,7 +90,8 @@ int tls_sk_attach(struct sock *sk, int optname, char __user *optval,
 		  unsigned int optlen);
 void tls_err_abort(struct sock *sk, int err);
 
-int tls_set_sw_offload(struct sock *sk, int tx);
+int tls_set_sw_offload(struct sock *sk, int tx,
+		       struct tls_crypto_info *new_crypto_info);
 void tls_update_rx_zc_capable(struct tls_context *tls_ctx);
 void tls_sw_strparser_arm(struct sock *sk, struct tls_context *ctx);
 void tls_sw_strparser_done(struct tls_context *tls_ctx);
net/tls/tls_device.c +1 -1
index c149f36b42ee..1ad50c253dfe 100644
@@ -1291,7 +1291,7 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
 	context->resync_nh_reset = 1;
 
 	ctx->priv_ctx_rx = context;
-	rc = tls_set_sw_offload(sk, 0);
+	rc = tls_set_sw_offload(sk, 0, NULL);
 	if (rc)
 		goto release_ctx;
 
net/tls/tls_main.c +27 -5
index fb1da1780f50..9be82aecd13e 100644
@@ -669,9 +669,12 @@ static int tls_getsockopt(struct sock *sk, int level, int optname,
 static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 				  unsigned int optlen, int tx)
 {
+	union tls_crypto_context tmp = {};
+	struct tls_crypto_info *old_crypto_info = NULL;
 	struct tls_crypto_info *crypto_info;
 	struct tls_crypto_info *alt_crypto_info;
 	struct tls_context *ctx = tls_get_ctx(sk);
+	bool update = false;
 	size_t optsize;
 	int rc = 0;
 	int conf;
@@ -687,9 +690,17 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 		alt_crypto_info = &ctx->crypto_send.info;
 	}
 
-	/* Currently we don't support set crypto info more than one time */
-	if (TLS_CRYPTO_INFO_READY(crypto_info))
-		return -EBUSY;
+	if (TLS_CRYPTO_INFO_READY(crypto_info)) {
+		/* Currently we only support setting crypto info more
+		 * than one time for TLS 1.3
+		 */
+		if (crypto_info->version != TLS_1_3_VERSION)
+			return -EBUSY;
+
+		update = true;
+		old_crypto_info = crypto_info;
+		crypto_info = &tmp.info;
+	}
 
 	rc = copy_from_sockptr(crypto_info, optval, sizeof(*crypto_info));
 	if (rc) {
@@ -704,6 +715,15 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 		goto err_crypto_info;
 	}
 
+	if (update) {
+		/* Ensure that TLS version and ciphers are not modified */
+		if (crypto_info->version != old_crypto_info->version ||
+		    crypto_info->cipher_type != old_crypto_info->cipher_type) {
+			rc = -EINVAL;
+			goto err_crypto_info;
+		}
+	}
+
 	/* Ensure that TLS version and ciphers are same in both directions */
 	if (TLS_CRYPTO_INFO_READY(alt_crypto_info)) {
 		if (alt_crypto_info->version != crypto_info->version ||
@@ -772,7 +792,8 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXDEVICE);
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXDEVICE);
 		} else {
-			rc = tls_set_sw_offload(sk, 1);
+			rc = tls_set_sw_offload(sk, 1,
+						update ? crypto_info : NULL);
 			if (rc)
 				goto err_crypto_info;
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXSW);
@@ -786,7 +807,8 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXDEVICE);
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXDEVICE);
 		} else {
-			rc = tls_set_sw_offload(sk, 0);
+			rc = tls_set_sw_offload(sk, 0,
+						update ? crypto_info : NULL);
 			if (rc)
 				goto err_crypto_info;
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXSW);
net/tls/tls_sw.c +90 -30

221 lines not on this page · raw message

Signature · 1 line
2.38.1
#6 lore raw

Re: [PATCH net-next 3/5] tls: implement rekey for TLS1.3

Hi,

Thanks for posting this series!
We were working on the same feature.
CC Apoorv from s2n team.

From:   Sabrina Dubroca <[email protected]>
Date:   Tue, 17 Jan 2023 14:45:29 +0100
Quoted 64 lines

This adds the possibility to change the key and IV when using TLS1.3. Changing the cipher or TLS version is not supported.

Once we have updated the RX key, we can unblock the receive side.

This change only affects tls_sw, since 1.3 offload isn't supported.

---
 net/tls/tls.h        |   3 +-
 net/tls/tls_device.c |   2 +-
 net/tls/tls_main.c   |  32 ++++++++++--
 net/tls/tls_sw.c     | 120 ++++++++++++++++++++++++++++++++-----------
 4 files changed, 120 insertions(+), 37 deletions(-)
net/tls/tls.h +2 -1
index 34d0fe814600..6f9c85eaa9c5 100644
@@ -90,7 +90,8 @@ int tls_sk_attach(struct sock *sk, int optname, char __user *optval,
 		  unsigned int optlen);
 void tls_err_abort(struct sock *sk, int err);
 
-int tls_set_sw_offload(struct sock *sk, int tx);
+int tls_set_sw_offload(struct sock *sk, int tx,
+		       struct tls_crypto_info *new_crypto_info);
 void tls_update_rx_zc_capable(struct tls_context *tls_ctx);
 void tls_sw_strparser_arm(struct sock *sk, struct tls_context *ctx);
 void tls_sw_strparser_done(struct tls_context *tls_ctx);
net/tls/tls_device.c +1 -1
index c149f36b42ee..1ad50c253dfe 100644
@@ -1291,7 +1291,7 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
 	context->resync_nh_reset = 1;
 
 	ctx->priv_ctx_rx = context;
-	rc = tls_set_sw_offload(sk, 0);
+	rc = tls_set_sw_offload(sk, 0, NULL);
 	if (rc)
 		goto release_ctx;
 
net/tls/tls_main.c +3 -0
index fb1da1780f50..9be82aecd13e 100644
@@ -669,9 +669,12 @@ static int tls_getsockopt(struct sock *sk, int level, int optname,
 static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 				  unsigned int optlen, int tx)
 {
+	union tls_crypto_context tmp = {};
+	struct tls_crypto_info *old_crypto_info = NULL;
 	struct tls_crypto_info *crypto_info;
 	struct tls_crypto_info *alt_crypto_info;
 	struct tls_context *ctx = tls_get_ctx(sk);
+	bool update = false;
 	size_t optsize;
 	int rc = 0;
 	int conf;
@@ -687,9 +690,17 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 		alt_crypto_info = &ctx->crypto_send.info;
 	}
 

net/tls/tls_main.c static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval, · line 699

Quoted 10 lines
net/tls/tls_main.c +7 -3
-	/* Currently we don't support set crypto info more than one time */
-	if (TLS_CRYPTO_INFO_READY(crypto_info))
-		return -EBUSY;
+	if (TLS_CRYPTO_INFO_READY(crypto_info)) {
+		/* Currently we only support setting crypto info more
+		 * than one time for TLS 1.3
+		 */
+		if (crypto_info->version != TLS_1_3_VERSION)
+			return -EBUSY;
+

Should we check this ?

                if (!tx && !key_update_pending)
                        return -EBUSY;

Otherwise we can set a new RX key even if the other end has not sent KeyUpdateRequest.

Quoted 12 lines
net/tls/tls_main.c +5 -0
+		update = true;
+		old_crypto_info = crypto_info;
+		crypto_info = &tmp.info;
+	}
 
 	rc = copy_from_sockptr(crypto_info, optval, sizeof(*crypto_info));
 	if (rc) {
@@ -704,6 +715,15 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 		goto err_crypto_info;
 	}
 
+	if (update) {

net/tls/tls_main.c static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval, · line 728

Quoted 10 lines
net/tls/tls_main.c +8 -0
+		/* Ensure that TLS version and ciphers are not modified */
+		if (crypto_info->version != old_crypto_info->version ||
+		    crypto_info->cipher_type != old_crypto_info->cipher_type) {
+			rc = -EINVAL;
+			goto err_crypto_info;
+		}
+	}
+
 	/* Ensure that TLS version and ciphers are same in both directions */
 	if (TLS_CRYPTO_INFO_READY(alt_crypto_info)) {

We can change this to else-if.

Quoted 54 lines
net/tls/tls_main.c +4 -2
 		if (alt_crypto_info->version != crypto_info->version ||
@@ -772,7 +792,8 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXDEVICE);
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXDEVICE);
 		} else {
-			rc = tls_set_sw_offload(sk, 1);
+			rc = tls_set_sw_offload(sk, 1,
+						update ? crypto_info : NULL);
 			if (rc)
 				goto err_crypto_info;
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXSW);
@@ -786,7 +807,8 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXDEVICE);
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXDEVICE);
 		} else {
-			rc = tls_set_sw_offload(sk, 0);
+			rc = tls_set_sw_offload(sk, 0,
+						update ? crypto_info : NULL);
 			if (rc)
 				goto err_crypto_info;
 			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXSW);
net/tls/tls_sw.c +11 -3
index 22efea224a04..310135aaa6e6 100644
@@ -2505,11 +2505,19 @@ void tls_update_rx_zc_capable(struct tls_context *tls_ctx)
 		tls_ctx->prot_info.version != TLS_1_3_VERSION;
 }
 
-int tls_set_sw_offload(struct sock *sk, int tx)
+static void tls_finish_key_update(struct tls_context *tls_ctx)
+{
+	struct tls_sw_context_rx *ctx = tls_ctx->priv_ctx_rx;
+
+	ctx->key_update_pending = false;
+}
+
+int tls_set_sw_offload(struct sock *sk, int tx,
+		       struct tls_crypto_info *new_crypto_info)
 {
 	struct tls_context *ctx = tls_get_ctx(sk);
 	struct tls_prot_info *prot = &ctx->prot_info;
-	struct tls_crypto_info *crypto_info;
+	struct tls_crypto_info *crypto_info, *src_crypto_info;
 	struct tls_sw_context_tx *sw_ctx_tx = NULL;
 	struct tls_sw_context_rx *sw_ctx_rx = NULL;
 	struct cipher_context *cctx;
@@ -2517,9 +2525,28 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	u16 nonce_size, tag_size, iv_size, rec_seq_size, salt_size;
 	struct crypto_tfm *tfm;
 	char *iv, *rec_seq, *key, *salt, *cipher_name;
-	size_t keysize;
+	size_t keysize, crypto_info_size;
 	int rc = 0;

net/tls/tls_sw.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2539

Quoted 10 lines
net/tls/tls_sw.c +9 -0
 
+	if (new_crypto_info) {
+		/* non-NULL new_crypto_info means rekey */
+		src_crypto_info = new_crypto_info;
+		if (tx) {
+			sw_ctx_tx = ctx->priv_ctx_tx;
+			crypto_info = &ctx->crypto_send.info;
+			cctx = &ctx->tx;
+			aead = &sw_ctx_tx->aead_send;
+			sw_ctx_tx = NULL;

sw_ctx_tx is already initialised.

net/tls/tls_sw.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2545

Quoted 6 lines
net/tls/tls_sw.c +6 -0
+		} else {
+			sw_ctx_rx = ctx->priv_ctx_rx;
+			crypto_info = &ctx->crypto_recv.info;
+			cctx = &ctx->rx;
+			aead = &sw_ctx_rx->aead_recv;
+			sw_ctx_rx = NULL;

Same here.

Quoted 192 lines
net/tls/tls_sw.c +64 -27
+		}
+		goto skip_init;
+	}
+
 	if (tx) {
 		if (!ctx->priv_ctx_tx) {
 			sw_ctx_tx = kzalloc(sizeof(*sw_ctx_tx), GFP_KERNEL);
@@ -2566,12 +2593,15 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 		aead = &sw_ctx_rx->aead_recv;
 		sw_ctx_rx->key_update_pending = false;
 	}
+	src_crypto_info = crypto_info;
 
+skip_init:
 	switch (crypto_info->cipher_type) {
 	case TLS_CIPHER_AES_GCM_128: {
 		struct tls12_crypto_info_aes_gcm_128 *gcm_128_info;
 
-		gcm_128_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aes_gcm_128);
+		gcm_128_info = (void *)src_crypto_info;
 		nonce_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
 		tag_size = TLS_CIPHER_AES_GCM_128_TAG_SIZE;
 		iv_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
@@ -2588,7 +2618,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	case TLS_CIPHER_AES_GCM_256: {
 		struct tls12_crypto_info_aes_gcm_256 *gcm_256_info;
 
-		gcm_256_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aes_gcm_256);
+		gcm_256_info = (void *)src_crypto_info;
 		nonce_size = TLS_CIPHER_AES_GCM_256_IV_SIZE;
 		tag_size = TLS_CIPHER_AES_GCM_256_TAG_SIZE;
 		iv_size = TLS_CIPHER_AES_GCM_256_IV_SIZE;
@@ -2605,7 +2636,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	case TLS_CIPHER_AES_CCM_128: {
 		struct tls12_crypto_info_aes_ccm_128 *ccm_128_info;
 
-		ccm_128_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aes_ccm_128);
+		ccm_128_info = (void *)src_crypto_info;
 		nonce_size = TLS_CIPHER_AES_CCM_128_IV_SIZE;
 		tag_size = TLS_CIPHER_AES_CCM_128_TAG_SIZE;
 		iv_size = TLS_CIPHER_AES_CCM_128_IV_SIZE;
@@ -2622,7 +2654,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	case TLS_CIPHER_CHACHA20_POLY1305: {
 		struct tls12_crypto_info_chacha20_poly1305 *chacha20_poly1305_info;
 
-		chacha20_poly1305_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_chacha20_poly1305);
+		chacha20_poly1305_info = (void *)src_crypto_info;
 		nonce_size = 0;
 		tag_size = TLS_CIPHER_CHACHA20_POLY1305_TAG_SIZE;
 		iv_size = TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE;
@@ -2639,7 +2672,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	case TLS_CIPHER_SM4_GCM: {
 		struct tls12_crypto_info_sm4_gcm *sm4_gcm_info;
 
-		sm4_gcm_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_sm4_gcm);
+		sm4_gcm_info = (void *)src_crypto_info;
 		nonce_size = TLS_CIPHER_SM4_GCM_IV_SIZE;
 		tag_size = TLS_CIPHER_SM4_GCM_TAG_SIZE;
 		iv_size = TLS_CIPHER_SM4_GCM_IV_SIZE;
@@ -2656,7 +2690,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	case TLS_CIPHER_SM4_CCM: {
 		struct tls12_crypto_info_sm4_ccm *sm4_ccm_info;
 
-		sm4_ccm_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_sm4_ccm);
+		sm4_ccm_info = (void *)src_crypto_info;
 		nonce_size = TLS_CIPHER_SM4_CCM_IV_SIZE;
 		tag_size = TLS_CIPHER_SM4_CCM_TAG_SIZE;
 		iv_size = TLS_CIPHER_SM4_CCM_IV_SIZE;
@@ -2673,7 +2708,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	case TLS_CIPHER_ARIA_GCM_128: {
 		struct tls12_crypto_info_aria_gcm_128 *aria_gcm_128_info;
 
-		aria_gcm_128_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aria_gcm_128);
+		aria_gcm_128_info = (void *)src_crypto_info;
 		nonce_size = TLS_CIPHER_ARIA_GCM_128_IV_SIZE;
 		tag_size = TLS_CIPHER_ARIA_GCM_128_TAG_SIZE;
 		iv_size = TLS_CIPHER_ARIA_GCM_128_IV_SIZE;
@@ -2690,7 +2726,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	case TLS_CIPHER_ARIA_GCM_256: {
 		struct tls12_crypto_info_aria_gcm_256 *gcm_256_info;
 
-		gcm_256_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aria_gcm_256);
+		gcm_256_info = (void *)src_crypto_info;
 		nonce_size = TLS_CIPHER_ARIA_GCM_256_IV_SIZE;
 		tag_size = TLS_CIPHER_ARIA_GCM_256_TAG_SIZE;
 		iv_size = TLS_CIPHER_ARIA_GCM_256_IV_SIZE;
@@ -2734,19 +2771,26 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 			      prot->tag_size + prot->tail_size;
 	prot->iv_size = iv_size;
 	prot->salt_size = salt_size;
-	cctx->iv = kmalloc(iv_size + salt_size, GFP_KERNEL);
-	if (!cctx->iv) {
-		rc = -ENOMEM;
-		goto free_priv;
+	if (!new_crypto_info) {
+		cctx->iv = kmalloc(iv_size + salt_size, GFP_KERNEL);
+		if (!cctx->iv) {
+			rc = -ENOMEM;
+			goto free_priv;
+		}
 	}
 	/* Note: 128 & 256 bit salt are the same size */
 	prot->rec_seq_size = rec_seq_size;
 	memcpy(cctx->iv, salt, salt_size);
 	memcpy(cctx->iv + salt_size, iv, iv_size);
-	cctx->rec_seq = kmemdup(rec_seq, rec_seq_size, GFP_KERNEL);
-	if (!cctx->rec_seq) {
-		rc = -ENOMEM;
-		goto free_iv;
+
+	if (!new_crypto_info) {
+		cctx->rec_seq = kmemdup(rec_seq, rec_seq_size, GFP_KERNEL);
+		if (!cctx->rec_seq) {
+			rc = -ENOMEM;
+			goto free_iv;
+		}
+	} else {
+		memcpy(cctx->rec_seq, rec_seq, rec_seq_size);
 	}
 
 	if (!*aead) {
@@ -2761,13 +2805,20 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	ctx->push_pending_record = tls_sw_push_pending_record;
 
 	rc = crypto_aead_setkey(*aead, key, keysize);
-
-	if (rc)
-		goto free_aead;
+	if (rc) {
+		if (new_crypto_info)
+			goto out;
+		else
+			goto free_aead;
+	}
 
 	rc = crypto_aead_setauthsize(*aead, prot->tag_size);
-	if (rc)
-		goto free_aead;
+	if (rc) {
+		if (new_crypto_info)
+			goto out;
+		else
+			goto free_aead;
+	}
 
 	if (sw_ctx_rx) {
 		tfm = crypto_aead_tfm(sw_ctx_rx->aead_recv);
@@ -2782,6 +2833,13 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 			goto free_aead;
 	}
 
+	if (new_crypto_info) {
+		memcpy(crypto_info, new_crypto_info, crypto_info_size);
+		memzero_explicit(new_crypto_info, crypto_info_size);
+		if (!tx)
+			tls_finish_key_update(ctx);
+	}
+
 	goto out;
 
 free_aead:
@@ -2794,12 +2852,14 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	kfree(cctx->iv);
 	cctx->iv = NULL;
 free_priv:
-	if (tx) {
-		kfree(ctx->priv_ctx_tx);
-		ctx->priv_ctx_tx = NULL;
-	} else {
-		kfree(ctx->priv_ctx_rx);
-		ctx->priv_ctx_rx = NULL;
+	if (!new_crypto_info) {
+		if (tx) {
+			kfree(ctx->priv_ctx_tx);
+			ctx->priv_ctx_tx = NULL;
+		} else {
+			kfree(ctx->priv_ctx_rx);
+			ctx->priv_ctx_rx = NULL;
+		}
 	}
 out:
 	return rc;
Signature · 1 line
2.38.1
maintainer
#15 lore raw
In reply to Sabrina Dubroca

Please CC all the maintainers.

Quoted 11 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 8 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Quoted 2 lines

Note: in a future series, I'll clean up tls_set_sw_offload and eliminate the per-cipher copy-paste using tls_cipher_size_desc.

Yeah, I think it's on Vadim's TODO list as well.

author maintainer
#16 lore raw
Quoted 1 line · Jakub Kicinski

Please CC all the maintainers.

Sorry.

Quoted 13 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:
Quoted 11 lines

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

Quoted 4 lines
Quoted 4 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

Quoted 8 lines
Quoted 4 lines

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

IIRC support for KeyUpdates is mandatory in TLS1.3, so currently the kernel can't claim to support 1.3, independent of offloading.

Some folks did tests with and without kTLS using nbdcopy and found a small but noticeable performance improvement (around 8-10%).

Quoted 4 lines
Quoted 2 lines

Note: in a future series, I'll clean up tls_set_sw_offload and eliminate the per-cipher copy-paste using tls_cipher_size_desc.

Yeah, I think it's on Vadim's TODO list as well.

I've already done most of the work as I was working on this, I'll submit it later.

Signature · 1 line
Sabrina
author maintainer
#7 lore raw

Re: [PATCH net-next 3/5] tls: implement rekey for TLS1.3

In reply to Kuniyuki Iwashima
Quoted 5 lines · Kuniyuki Iwashima

Hi,

Thanks for posting this series!
We were working on the same feature.
CC Apoorv from s2n team.

Ah, cool. Does the behavior in those patches match what your implementation?

[...]

Quoted 34 lines
Quoted 21 lines
net/tls/tls_main.c +3 -0
index fb1da1780f50..9be82aecd13e 100644
@@ -669,9 +669,12 @@ static int tls_getsockopt(struct sock *sk, int level, int optname,
 static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 				  unsigned int optlen, int tx)
 {
+	union tls_crypto_context tmp = {};
+	struct tls_crypto_info *old_crypto_info = NULL;
 	struct tls_crypto_info *crypto_info;
 	struct tls_crypto_info *alt_crypto_info;
 	struct tls_context *ctx = tls_get_ctx(sk);
+	bool update = false;
 	size_t optsize;
 	int rc = 0;
 	int conf;
@@ -687,9 +690,17 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 		alt_crypto_info = &ctx->crypto_send.info;
 	}
 

net/tls/tls_main.c static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval, · line 699

Quoted 10 lines
net/tls/tls_main.c +7 -3
-	/* Currently we don't support set crypto info more than one time */
-	if (TLS_CRYPTO_INFO_READY(crypto_info))
-		return -EBUSY;
+	if (TLS_CRYPTO_INFO_READY(crypto_info)) {
+		/* Currently we only support setting crypto info more
+		 * than one time for TLS 1.3
+		 */
+		if (crypto_info->version != TLS_1_3_VERSION)
+			return -EBUSY;
+

Should we check this ?

Quoted 5 lines
                if (!tx && !key_update_pending)
                        return -EBUSY;

Otherwise we can set a new RX key even if the other end has not sent KeyUpdateRequest.

Maybe. My thinking was "let userspace shoot itself in the foot if it wants".

Quoted 11 lines
Quoted 11 lines
net/tls/tls_main.c +4 -0
+		update = true;
+		old_crypto_info = crypto_info;
+		crypto_info = &tmp.info;
+	}
 
 	rc = copy_from_sockptr(crypto_info, optval, sizeof(*crypto_info));
 	if (rc) {
@@ -704,6 +715,15 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 		goto err_crypto_info;
 	}
 
Quoted 13 lines

net/tls/tls_main.c static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval, · line 728

Quoted 11 lines
net/tls/tls_main.c +9 -0
+	if (update) {
+		/* Ensure that TLS version and ciphers are not modified */
+		if (crypto_info->version != old_crypto_info->version ||
+		    crypto_info->cipher_type != old_crypto_info->cipher_type) {
+			rc = -EINVAL;
+			goto err_crypto_info;
+		}
+	}
+
 	/* Ensure that TLS version and ciphers are same in both directions */
 	if (TLS_CRYPTO_INFO_READY(alt_crypto_info)) {

We can change this to else-if.

Ok.

Quoted 1 line
Quoted 1 line
net/tls/tls_main.c +0 -0
 		if (alt_crypto_info->version != crypto_info->version ||

[...]

Quoted 8 lines
Quoted 8 lines
net/tls/tls_main.c +1 -1
@@ -2517,9 +2525,28 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	u16 nonce_size, tag_size, iv_size, rec_seq_size, salt_size;
 	struct crypto_tfm *tfm;
 	char *iv, *rec_seq, *key, *salt, *cipher_name;
-	size_t keysize;
+	size_t keysize, crypto_info_size;
 	int rc = 0;
 
Quoted 11 lines

net/tls/tls_main.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2539

Quoted 9 lines
net/tls/tls_main.c +9 -0
+	if (new_crypto_info) {
+		/* non-NULL new_crypto_info means rekey */
+		src_crypto_info = new_crypto_info;
+		if (tx) {
+			sw_ctx_tx = ctx->priv_ctx_tx;
+			crypto_info = &ctx->crypto_send.info;
+			cctx = &ctx->tx;
+			aead = &sw_ctx_tx->aead_send;
+			sw_ctx_tx = NULL;

sw_ctx_tx is already initialised.

No, it was NULL at the beginning of the function, but then I used it to set aead on the previous line, so I need to clear it again. I could use a temp variable instead if you think it's better.

Quoted 10 lines

net/tls/tls_main.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2545

Quoted 6 lines
net/tls/tls_main.c +6 -0
+		} else {
+			sw_ctx_rx = ctx->priv_ctx_rx;
+			crypto_info = &ctx->crypto_recv.info;
+			cctx = &ctx->rx;
+			aead = &sw_ctx_rx->aead_recv;
+			sw_ctx_rx = NULL;

Same here.

Quoted 7 lines

net/tls/tls_main.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2552

Quoted 7 lines
net/tls/tls_main.c +4 -0
+		}
+		goto skip_init;
+	}
+
 	if (tx) {
 		if (!ctx->priv_ctx_tx) {
 			sw_ctx_tx = kzalloc(sizeof(*sw_ctx_tx), GFP_KERNEL);

Thanks for the comments.

Signature · 1 line
Sabrina
Reviewed-by on v1 1/5
#2 lore raw

Re: [PATCH net-next 1/5] tls: remove tls_context argument from tls_set_sw_offload

In reply to Sabrina Dubroca
Quoted 66 lines · Sabrina Dubroca

It's not really needed since we end up refetching it as tls_ctx. We can also remove the NULL check, since we have already dereferenced ctx in do_tls_setsockopt_conf.

---
  net/tls/tls.h        |  2 +-
  net/tls/tls_device.c |  2 +-
  net/tls/tls_main.c   |  4 ++--
  net/tls/tls_sw.c     | 11 +++--------
  4 files changed, 7 insertions(+), 12 deletions(-)
net/tls/tls.h +1 -1
index 0e840a0c3437..34d0fe814600 100644
@@ -90,7 +90,7 @@ int tls_sk_attach(struct sock *sk, int optname, char __user *optval,
  		  unsigned int optlen);
  void tls_err_abort(struct sock *sk, int err);
 
-int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx);
+int tls_set_sw_offload(struct sock *sk, int tx);
  void tls_update_rx_zc_capable(struct tls_context *tls_ctx);
  void tls_sw_strparser_arm(struct sock *sk, struct tls_context *ctx);
  void tls_sw_strparser_done(struct tls_context *tls_ctx);
net/tls/tls_device.c +1 -1
index 6c593788dc25..c149f36b42ee 100644
@@ -1291,7 +1291,7 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
  	context->resync_nh_reset = 1;
 
  	ctx->priv_ctx_rx = context;
-	rc = tls_set_sw_offload(sk, ctx, 0);
+	rc = tls_set_sw_offload(sk, 0);
  	if (rc)
  		goto release_ctx;
 
net/tls/tls_main.c +2 -2
index 3735cb00905d..fb1da1780f50 100644
@@ -772,7 +772,7 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXDEVICE);
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXDEVICE);
  		} else {
-			rc = tls_set_sw_offload(sk, ctx, 1);
+			rc = tls_set_sw_offload(sk, 1);
  			if (rc)
  				goto err_crypto_info;
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXSW);
@@ -786,7 +786,7 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXDEVICE);
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXDEVICE);
  		} else {
-			rc = tls_set_sw_offload(sk, ctx, 0);
+			rc = tls_set_sw_offload(sk, 0);
  			if (rc)
  				goto err_crypto_info;
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXSW);
net/tls/tls_sw.c +0 -0
index 9ed978634125..238562f9081b 100644

net/tls/tls_sw.c void tls_update_rx_zc_capable(struct tls_context *tls_ctx) · line 2468

Quoted 10 lines
net/tls/tls_sw.c +3 -3
@@ -2462,10 +2462,10 @@ void tls_update_rx_zc_capable(struct tls_context *tls_ctx)
  		tls_ctx->prot_info.version != TLS_1_3_VERSION;
  }
 
-int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
+int tls_set_sw_offload(struct sock *sk, int tx)
  {
-	struct tls_context *tls_ctx = tls_get_ctx(sk);
-	struct tls_prot_info *prot = &tls_ctx->prot_info;
+	struct tls_context *ctx = tls_get_ctx(sk);
+	struct tls_prot_info *prot = &ctx->prot_info;

nit: while you are here it's good idea to rearrange variables to follow reverse xmas tree rule

Quoted 5 lines
net/tls/tls_sw.c +0 -0
  	struct tls_crypto_info *crypto_info;
  	struct tls_sw_context_tx *sw_ctx_tx = NULL;
  	struct tls_sw_context_rx *sw_ctx_rx = NULL;
@@ -2477,11 +2477,6 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
  	size_t keysize;

net/tls/tls_sw.c int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx) · line 2482

Quoted 10 lines
net/tls/tls_sw.c +0 -5
  	int rc = 0;
 
-	if (!ctx) {
-		rc = -EINVAL;
-		goto out;
-	}
-
  	if (tx) {
  		if (!ctx->priv_ctx_tx) {
  			sw_ctx_tx = kzalloc(sizeof(*sw_ctx_tx), GFP_KERNEL);

we may consider changing tls_set_device_offload{,_rx} the same for consistency.

#10 lore raw

Re: [PATCH net-next 3/5] tls: implement rekey for TLS1.3

In reply to Sabrina Dubroca
Quoted 84 lines · Sabrina Dubroca

This adds the possibility to change the key and IV when using TLS1.3. Changing the cipher or TLS version is not supported.

Once we have updated the RX key, we can unblock the receive side.

This change only affects tls_sw, since 1.3 offload isn't supported.

---
  net/tls/tls.h        |   3 +-
  net/tls/tls_device.c |   2 +-
  net/tls/tls_main.c   |  32 ++++++++++--
  net/tls/tls_sw.c     | 120 ++++++++++++++++++++++++++++++++-----------
  4 files changed, 120 insertions(+), 37 deletions(-)
net/tls/tls.h +2 -1
index 34d0fe814600..6f9c85eaa9c5 100644
@@ -90,7 +90,8 @@ int tls_sk_attach(struct sock *sk, int optname, char __user *optval,
  		  unsigned int optlen);
  void tls_err_abort(struct sock *sk, int err);
 
-int tls_set_sw_offload(struct sock *sk, int tx);
+int tls_set_sw_offload(struct sock *sk, int tx,
+		       struct tls_crypto_info *new_crypto_info);
  void tls_update_rx_zc_capable(struct tls_context *tls_ctx);
  void tls_sw_strparser_arm(struct sock *sk, struct tls_context *ctx);
  void tls_sw_strparser_done(struct tls_context *tls_ctx);
net/tls/tls_device.c +1 -1
index c149f36b42ee..1ad50c253dfe 100644
@@ -1291,7 +1291,7 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
  	context->resync_nh_reset = 1;
 
  	ctx->priv_ctx_rx = context;
-	rc = tls_set_sw_offload(sk, 0);
+	rc = tls_set_sw_offload(sk, 0, NULL);
  	if (rc)
  		goto release_ctx;
 
net/tls/tls_main.c +14 -3
index fb1da1780f50..9be82aecd13e 100644
@@ -669,9 +669,12 @@ static int tls_getsockopt(struct sock *sk, int level, int optname,
  static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
  				  unsigned int optlen, int tx)
  {
+	union tls_crypto_context tmp = {};
+	struct tls_crypto_info *old_crypto_info = NULL;
  	struct tls_crypto_info *crypto_info;
  	struct tls_crypto_info *alt_crypto_info;
  	struct tls_context *ctx = tls_get_ctx(sk);
+	bool update = false;
  	size_t optsize;
  	int rc = 0;
  	int conf;
@@ -687,9 +690,17 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
  		alt_crypto_info = &ctx->crypto_send.info;
  	}
 
-	/* Currently we don't support set crypto info more than one time */
-	if (TLS_CRYPTO_INFO_READY(crypto_info))
-		return -EBUSY;
+	if (TLS_CRYPTO_INFO_READY(crypto_info)) {
+		/* Currently we only support setting crypto info more
+		 * than one time for TLS 1.3
+		 */
+		if (crypto_info->version != TLS_1_3_VERSION)
+			return -EBUSY;
+
+		update = true;
+		old_crypto_info = crypto_info;
+		crypto_info = &tmp.info;
+	}
 
  	rc = copy_from_sockptr(crypto_info, optval, sizeof(*crypto_info));
  	if (rc) {
@@ -704,6 +715,15 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
  		goto err_crypto_info;
  	}

net/tls/tls_main.c static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval, · line 726

Quoted 10 lines
net/tls/tls_main.c +9 -0
 
+	if (update) {
+		/* Ensure that TLS version and ciphers are not modified */
+		if (crypto_info->version != old_crypto_info->version ||
+		    crypto_info->cipher_type != old_crypto_info->cipher_type) {
+			rc = -EINVAL;
+			goto err_crypto_info;
+		}
+	}
+

looks like these checks can be moved up to TLS_CRYPTO_INFO_READY scope and there will be no need for extra variables.

Quoted 252 lines
net/tls/tls_main.c +4 -2
  	/* Ensure that TLS version and ciphers are same in both directions */
  	if (TLS_CRYPTO_INFO_READY(alt_crypto_info)) {
  		if (alt_crypto_info->version != crypto_info->version ||
@@ -772,7 +792,8 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXDEVICE);
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXDEVICE);
  		} else {
-			rc = tls_set_sw_offload(sk, 1);
+			rc = tls_set_sw_offload(sk, 1,
+						update ? crypto_info : NULL);
  			if (rc)
  				goto err_crypto_info;
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXSW);
@@ -786,7 +807,8 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXDEVICE);
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXDEVICE);
  		} else {
-			rc = tls_set_sw_offload(sk, 0);
+			rc = tls_set_sw_offload(sk, 0,
+						update ? crypto_info : NULL);
  			if (rc)
  				goto err_crypto_info;
  			TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXSW);
net/tls/tls_sw.c +83 -30
index 22efea224a04..310135aaa6e6 100644
@@ -2505,11 +2505,19 @@ void tls_update_rx_zc_capable(struct tls_context *tls_ctx)
  		tls_ctx->prot_info.version != TLS_1_3_VERSION;
  }
 
-int tls_set_sw_offload(struct sock *sk, int tx)
+static void tls_finish_key_update(struct tls_context *tls_ctx)
+{
+	struct tls_sw_context_rx *ctx = tls_ctx->priv_ctx_rx;
+
+	ctx->key_update_pending = false;
+}
+
+int tls_set_sw_offload(struct sock *sk, int tx,
+		       struct tls_crypto_info *new_crypto_info)
  {
  	struct tls_context *ctx = tls_get_ctx(sk);
  	struct tls_prot_info *prot = &ctx->prot_info;
-	struct tls_crypto_info *crypto_info;
+	struct tls_crypto_info *crypto_info, *src_crypto_info;
  	struct tls_sw_context_tx *sw_ctx_tx = NULL;
  	struct tls_sw_context_rx *sw_ctx_rx = NULL;
  	struct cipher_context *cctx;
@@ -2517,9 +2525,28 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	u16 nonce_size, tag_size, iv_size, rec_seq_size, salt_size;
  	struct crypto_tfm *tfm;
  	char *iv, *rec_seq, *key, *salt, *cipher_name;
-	size_t keysize;
+	size_t keysize, crypto_info_size;
  	int rc = 0;
 
+	if (new_crypto_info) {
+		/* non-NULL new_crypto_info means rekey */
+		src_crypto_info = new_crypto_info;
+		if (tx) {
+			sw_ctx_tx = ctx->priv_ctx_tx;
+			crypto_info = &ctx->crypto_send.info;
+			cctx = &ctx->tx;
+			aead = &sw_ctx_tx->aead_send;
+			sw_ctx_tx = NULL;
+		} else {
+			sw_ctx_rx = ctx->priv_ctx_rx;
+			crypto_info = &ctx->crypto_recv.info;
+			cctx = &ctx->rx;
+			aead = &sw_ctx_rx->aead_recv;
+			sw_ctx_rx = NULL;
+		}
+		goto skip_init;
+	}
+
  	if (tx) {
  		if (!ctx->priv_ctx_tx) {
  			sw_ctx_tx = kzalloc(sizeof(*sw_ctx_tx), GFP_KERNEL);
@@ -2566,12 +2593,15 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  		aead = &sw_ctx_rx->aead_recv;
  		sw_ctx_rx->key_update_pending = false;
  	}
+	src_crypto_info = crypto_info;
 
+skip_init:
  	switch (crypto_info->cipher_type) {
  	case TLS_CIPHER_AES_GCM_128: {
  		struct tls12_crypto_info_aes_gcm_128 *gcm_128_info;
 
-		gcm_128_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aes_gcm_128);
+		gcm_128_info = (void *)src_crypto_info;
  		nonce_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
  		tag_size = TLS_CIPHER_AES_GCM_128_TAG_SIZE;
  		iv_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
@@ -2588,7 +2618,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	case TLS_CIPHER_AES_GCM_256: {
  		struct tls12_crypto_info_aes_gcm_256 *gcm_256_info;
 
-		gcm_256_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aes_gcm_256);
+		gcm_256_info = (void *)src_crypto_info;
  		nonce_size = TLS_CIPHER_AES_GCM_256_IV_SIZE;
  		tag_size = TLS_CIPHER_AES_GCM_256_TAG_SIZE;
  		iv_size = TLS_CIPHER_AES_GCM_256_IV_SIZE;
@@ -2605,7 +2636,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	case TLS_CIPHER_AES_CCM_128: {
  		struct tls12_crypto_info_aes_ccm_128 *ccm_128_info;
 
-		ccm_128_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aes_ccm_128);
+		ccm_128_info = (void *)src_crypto_info;
  		nonce_size = TLS_CIPHER_AES_CCM_128_IV_SIZE;
  		tag_size = TLS_CIPHER_AES_CCM_128_TAG_SIZE;
  		iv_size = TLS_CIPHER_AES_CCM_128_IV_SIZE;
@@ -2622,7 +2654,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	case TLS_CIPHER_CHACHA20_POLY1305: {
  		struct tls12_crypto_info_chacha20_poly1305 *chacha20_poly1305_info;
 
-		chacha20_poly1305_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_chacha20_poly1305);
+		chacha20_poly1305_info = (void *)src_crypto_info;
  		nonce_size = 0;
  		tag_size = TLS_CIPHER_CHACHA20_POLY1305_TAG_SIZE;
  		iv_size = TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE;
@@ -2639,7 +2672,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	case TLS_CIPHER_SM4_GCM: {
  		struct tls12_crypto_info_sm4_gcm *sm4_gcm_info;
 
-		sm4_gcm_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_sm4_gcm);
+		sm4_gcm_info = (void *)src_crypto_info;
  		nonce_size = TLS_CIPHER_SM4_GCM_IV_SIZE;
  		tag_size = TLS_CIPHER_SM4_GCM_TAG_SIZE;
  		iv_size = TLS_CIPHER_SM4_GCM_IV_SIZE;
@@ -2656,7 +2690,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	case TLS_CIPHER_SM4_CCM: {
  		struct tls12_crypto_info_sm4_ccm *sm4_ccm_info;
 
-		sm4_ccm_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_sm4_ccm);
+		sm4_ccm_info = (void *)src_crypto_info;
  		nonce_size = TLS_CIPHER_SM4_CCM_IV_SIZE;
  		tag_size = TLS_CIPHER_SM4_CCM_TAG_SIZE;
  		iv_size = TLS_CIPHER_SM4_CCM_IV_SIZE;
@@ -2673,7 +2708,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	case TLS_CIPHER_ARIA_GCM_128: {
  		struct tls12_crypto_info_aria_gcm_128 *aria_gcm_128_info;
 
-		aria_gcm_128_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aria_gcm_128);
+		aria_gcm_128_info = (void *)src_crypto_info;
  		nonce_size = TLS_CIPHER_ARIA_GCM_128_IV_SIZE;
  		tag_size = TLS_CIPHER_ARIA_GCM_128_TAG_SIZE;
  		iv_size = TLS_CIPHER_ARIA_GCM_128_IV_SIZE;
@@ -2690,7 +2726,8 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	case TLS_CIPHER_ARIA_GCM_256: {
  		struct tls12_crypto_info_aria_gcm_256 *gcm_256_info;
 
-		gcm_256_info = (void *)crypto_info;
+		crypto_info_size = sizeof(struct tls12_crypto_info_aria_gcm_256);
+		gcm_256_info = (void *)src_crypto_info;
  		nonce_size = TLS_CIPHER_ARIA_GCM_256_IV_SIZE;
  		tag_size = TLS_CIPHER_ARIA_GCM_256_TAG_SIZE;
  		iv_size = TLS_CIPHER_ARIA_GCM_256_IV_SIZE;
@@ -2734,19 +2771,26 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  			      prot->tag_size + prot->tail_size;
  	prot->iv_size = iv_size;
  	prot->salt_size = salt_size;
-	cctx->iv = kmalloc(iv_size + salt_size, GFP_KERNEL);
-	if (!cctx->iv) {
-		rc = -ENOMEM;
-		goto free_priv;
+	if (!new_crypto_info) {
+		cctx->iv = kmalloc(iv_size + salt_size, GFP_KERNEL);
+		if (!cctx->iv) {
+			rc = -ENOMEM;
+			goto free_priv;
+		}
  	}
  	/* Note: 128 & 256 bit salt are the same size */
  	prot->rec_seq_size = rec_seq_size;
  	memcpy(cctx->iv, salt, salt_size);
  	memcpy(cctx->iv + salt_size, iv, iv_size);
-	cctx->rec_seq = kmemdup(rec_seq, rec_seq_size, GFP_KERNEL);
-	if (!cctx->rec_seq) {
-		rc = -ENOMEM;
-		goto free_iv;
+
+	if (!new_crypto_info) {
+		cctx->rec_seq = kmemdup(rec_seq, rec_seq_size, GFP_KERNEL);
+		if (!cctx->rec_seq) {
+			rc = -ENOMEM;
+			goto free_iv;
+		}
+	} else {
+		memcpy(cctx->rec_seq, rec_seq, rec_seq_size);
  	}
 
  	if (!*aead) {
@@ -2761,13 +2805,20 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	ctx->push_pending_record = tls_sw_push_pending_record;
 
  	rc = crypto_aead_setkey(*aead, key, keysize);
-
-	if (rc)
-		goto free_aead;
+	if (rc) {
+		if (new_crypto_info)
+			goto out;
+		else
+			goto free_aead;
+	}
 
  	rc = crypto_aead_setauthsize(*aead, prot->tag_size);
-	if (rc)
-		goto free_aead;
+	if (rc) {
+		if (new_crypto_info)
+			goto out;
+		else
+			goto free_aead;
+	}
 
  	if (sw_ctx_rx) {
  		tfm = crypto_aead_tfm(sw_ctx_rx->aead_recv);
@@ -2782,6 +2833,13 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  			goto free_aead;
  	}
 
+	if (new_crypto_info) {
+		memcpy(crypto_info, new_crypto_info, crypto_info_size);
+		memzero_explicit(new_crypto_info, crypto_info_size);
+		if (!tx)
+			tls_finish_key_update(ctx);
+	}
+
  	goto out;
 
  free_aead:
@@ -2794,12 +2852,14 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	kfree(cctx->iv);
  	cctx->iv = NULL;
  free_priv:
-	if (tx) {
-		kfree(ctx->priv_ctx_tx);
-		ctx->priv_ctx_tx = NULL;
-	} else {
-		kfree(ctx->priv_ctx_rx);
-		ctx->priv_ctx_rx = NULL;
+	if (!new_crypto_info) {

net/tls/tls_sw.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2865

Quoted 10 lines
net/tls/tls_sw.c +7 -0
+		if (tx) {
+			kfree(ctx->priv_ctx_tx);
+			ctx->priv_ctx_tx = NULL;
+		} else {
+			kfree(ctx->priv_ctx_rx);
+			ctx->priv_ctx_rx = NULL;
+		}
  	}
  out:
  	return rc;

I think we can avoid extra parameter and extra level of if{} constructions by checking if iv and rec_seq is already allocated and adjust init part the same way. I don't think we have to have separate error path because in case of any error during rekey procedure the connection becomes useless and application should indicate error to the other end. The code copies new crypto info to the current storage, so it assumes that all fields a properly filled and that means that this copy can be done earlier and use the same code path as first init code.

#8 lore raw

Re: [PATCH net-next 3/5] tls: implement rekey for TLS1.3

In reply to Sabrina Dubroca
Quoted 9 lines
Quoted 5 lines · Kuniyuki Iwashima

Hi,

Thanks for posting this series!
We were working on the same feature.
CC Apoorv from s2n team.

Ah, cool. Does the behavior in those patches match what your implementation?

Thanks for submitting this, it looks great! We are working on testing this now.

Quoted 36 lines

[...]

Quoted 34 lines
Quoted 21 lines
net/tls/tls_main.c +3 -0
index fb1da1780f50..9be82aecd13e 100644
@@ -669,9 +669,12 @@ static int tls_getsockopt(struct sock *sk, int level, int optname,
 static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 				  unsigned int optlen, int tx)
 {
+	union tls_crypto_context tmp = {};
+	struct tls_crypto_info *old_crypto_info = NULL;
 	struct tls_crypto_info *crypto_info;
 	struct tls_crypto_info *alt_crypto_info;
 	struct tls_context *ctx = tls_get_ctx(sk);
+	bool update = false;
 	size_t optsize;
 	int rc = 0;
 	int conf;
@@ -687,9 +690,17 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 		alt_crypto_info = &ctx->crypto_send.info;
 	}
 

net/tls/tls_main.c static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval, · line 699

Quoted 10 lines
net/tls/tls_main.c +7 -3
-	/* Currently we don't support set crypto info more than one time */
-	if (TLS_CRYPTO_INFO_READY(crypto_info))
-		return -EBUSY;
+	if (TLS_CRYPTO_INFO_READY(crypto_info)) {
+		/* Currently we only support setting crypto info more
+		 * than one time for TLS 1.3
+		 */
+		if (crypto_info->version != TLS_1_3_VERSION)
+			return -EBUSY;
+

Should we check this ?

Quoted 8 lines
Quoted 5 lines
                if (!tx && !key_update_pending)
                        return -EBUSY;

Otherwise we can set a new RX key even if the other end has not sent KeyUpdateRequest.

Maybe. My thinking was "let userspace shoot itself in the foot if it wants".

I feel avoiding foot-guns is probably the correct thing to do. The RFC also has a requirement that re-key(process messages with new key) should only happen after a KeyUpdate is received so it would be nice if the kTLS implemention can help enforce this.

Based on the RFC https://www.rfc-editor.org/rfc/rfc8446#section-4.6.3:
Additionally, both sides MUST enforce that a KeyUpdate with the old key is received before accepting any messages encrypted with the new key. Failure to do so may allow message truncation attacks.

Quoted 64 lines
Quoted 12 lines
Quoted 12 lines
net/tls/tls_main.c +4 -0
+		update = true;
+		old_crypto_info = crypto_info;
+		crypto_info = &tmp.info;
+	}
 
 	rc = copy_from_sockptr(crypto_info, optval, sizeof(*crypto_info));
 	if (rc) {
@@ -704,6 +715,15 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 		goto err_crypto_info;
 	}
 
Quoted 13 lines

net/tls/tls_main.c static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval, · line 728

Quoted 11 lines
net/tls/tls_main.c +9 -0
+	if (update) {
+		/* Ensure that TLS version and ciphers are not modified */
+		if (crypto_info->version != old_crypto_info->version ||
+		    crypto_info->cipher_type != old_crypto_info->cipher_type) {
+			rc = -EINVAL;
+			goto err_crypto_info;
+		}
+	}
+
 	/* Ensure that TLS version and ciphers are same in both directions */
 	if (TLS_CRYPTO_INFO_READY(alt_crypto_info)) {

We can change this to else-if.

Ok.

Quoted 1 line
Quoted 1 line
net/tls/tls_main.c +0 -0
 		if (alt_crypto_info->version != crypto_info->version ||

[...]

Quoted 8 lines
Quoted 8 lines
net/tls/tls_main.c +1 -1
@@ -2517,9 +2525,28 @@ int tls_set_sw_offload(struct sock *sk, int tx)
 	u16 nonce_size, tag_size, iv_size, rec_seq_size, salt_size;
 	struct crypto_tfm *tfm;
 	char *iv, *rec_seq, *key, *salt, *cipher_name;
-	size_t keysize;
+	size_t keysize, crypto_info_size;
 	int rc = 0;
 
Quoted 11 lines

net/tls/tls_main.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2539

Quoted 9 lines
net/tls/tls_main.c +9 -0
+	if (new_crypto_info) {
+		/* non-NULL new_crypto_info means rekey */
+		src_crypto_info = new_crypto_info;
+		if (tx) {
+			sw_ctx_tx = ctx->priv_ctx_tx;
+			crypto_info = &ctx->crypto_send.info;
+			cctx = &ctx->tx;
+			aead = &sw_ctx_tx->aead_send;
+			sw_ctx_tx = NULL;

sw_ctx_tx is already initialised.

No, it was NULL at the beginning of the function, but then I used it to set aead on the previous line, so I need to clear it again. I could use a temp variable instead if you think it's better.

Quoted 10 lines

net/tls/tls_main.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2545

Quoted 6 lines
net/tls/tls_main.c +6 -0
+		} else {
+			sw_ctx_rx = ctx->priv_ctx_rx;
+			crypto_info = &ctx->crypto_recv.info;
+			cctx = &ctx->rx;
+			aead = &sw_ctx_rx->aead_recv;
+			sw_ctx_rx = NULL;

Same here.

Quoted 12 lines
Quoted 7 lines

net/tls/tls_main.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2552

Quoted 7 lines
net/tls/tls_main.c +4 -0
+		}
+		goto skip_init;
+	}
+
 	if (tx) {
 		if (!ctx->priv_ctx_tx) {
 			sw_ctx_tx = kzalloc(sizeof(*sw_ctx_tx), GFP_KERNEL);

Thanks for the comments.

Signature · 1 line
Sabrina

--
Apoorv

#4 lore raw
In reply to Sabrina Dubroca
Quoted 21 lines
Quoted 1 line · Jakub Kicinski

Please CC all the maintainers.

Sorry.

Quoted 13 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:
Quoted 11 lines

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 6 lines

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

Pausing also better aligns with the RFC also since all subsequent messages should be encrypted with the new key.

From the RFC https://www.rfc-editor.org/rfc/rfc8446#section-4.6.3
After sending a KeyUpdate message, the sender SHALL send all its traffic using the next generation of keys, computed as described in Section 7.2. Upon receiving a KeyUpdate, the receiver MUST update its receiving keys.

Quoted 23 lines
Quoted 5 lines
Quoted 5 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

Quoted 8 lines
Quoted 4 lines

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

IIRC support for KeyUpdates is mandatory in TLS1.3, so currently the kernel can't claim to support 1.3, independent of offloading.

Some folks did tests with and without kTLS using nbdcopy and found a small but noticeable performance improvement (around 8-10%).

Quoted 10 lines
Quoted 4 lines
Quoted 2 lines

Note: in a future series, I'll clean up tls_set_sw_offload and eliminate the per-cipher copy-paste using tls_cipher_size_desc.

Yeah, I think it's on Vadim's TODO list as well.

I've already done most of the work as I was working on this, I'll submit it later.

Signature · 1 line
Sabrina

--
Apoorv

maintainer
#17 lore raw
In reply to Sabrina Dubroca
Quoted 17 lines · On Wed, 18 Jan 2023 11:06:25 +0100 Sabrina Dubroca wrote:
Quoted 13 lines · Jakub Kicinski
Quoted 11 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 6 lines

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

IDK, we don't interpret any other content types/cmsgs, and for well behaved user space there should be no problem (right?). I'm weakly against, if nobody agrees with me you can keep as is.

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Quoted 6 lines
Quoted 3 lines

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

I'm pretty sure some devices support it. None of the vendors could be bothered to plumb in the kernel support, yet, tho.
I don't know of anyone supporting rekeying.

Quoted 2 lines

IIRC support for KeyUpdates is mandatory in TLS1.3, so currently the kernel can't claim to support 1.3, independent of offloading.

The problem is that we will not be able to rekey offloaded connections. For Tx it's a non-trivial problem given the current architecture. The offload is supposed to be transparent, we can't fail the rekey just because the TLS gotten offloaded.

Quoted 2 lines

Some folks did tests with and without kTLS using nbdcopy and found a small but noticeable performance improvement (around 8-10%).

#18 lore raw
Quoted 42 lines · Jakub Kicinski
Quoted 17 lines · On Wed, 18 Jan 2023 11:06:25 +0100 Sabrina Dubroca wrote:
Quoted 13 lines · Jakub Kicinski
Quoted 11 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 6 lines

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

IDK, we don't interpret any other content types/cmsgs, and for well behaved user space there should be no problem (right?). I'm weakly against, if nobody agrees with me you can keep as is.

Quoted 13 lines
Quoted 12 lines
Quoted 8 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Quoted 5 lines
Quoted 2 lines

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

I'm pretty sure some devices support it. None of the vendors could be bothered to plumb in the kernel support, yet, tho.

Our device supports TLS 1.3, it's in our plans to add driver/kernel support.

Quoted 1 line

I don't know of anyone supporting rekeying.

Boris, Tariq, do you know?

author maintainer
#11 lore raw

Re: [PATCH net-next 3/5] tls: implement rekey for TLS1.3

In reply to Vadim Fedorenko
Quoted 27 lines · Vadim Fedorenko
Quoted 26 lines · Sabrina Dubroca
net/tls/tls_main.c +15 -3
@@ -687,9 +690,17 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
  		alt_crypto_info = &ctx->crypto_send.info;
  	}
-	/* Currently we don't support set crypto info more than one time */
-	if (TLS_CRYPTO_INFO_READY(crypto_info))
-		return -EBUSY;
+	if (TLS_CRYPTO_INFO_READY(crypto_info)) {
+		/* Currently we only support setting crypto info more
+		 * than one time for TLS 1.3
+		 */
+		if (crypto_info->version != TLS_1_3_VERSION)
+			return -EBUSY;
+
+		update = true;
+		old_crypto_info = crypto_info;
+		crypto_info = &tmp.info;
+	}
  	rc = copy_from_sockptr(crypto_info, optval, sizeof(*crypto_info));
  	if (rc) {
@@ -704,6 +715,15 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
  		goto err_crypto_info;
  	}
+	if (update) {
+		/* Ensure that TLS version and ciphers are not modified */
+		if (crypto_info->version != old_crypto_info->version ||
+		    crypto_info->cipher_type != old_crypto_info->cipher_type) {
Quoted 8 lines

net/tls/tls_main.c static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval, · line 726

Quoted 5 lines
net/tls/tls_main.c +5 -0
+			rc = -EINVAL;
+			goto err_crypto_info;
+		}
+	}
+

looks like these checks can be moved up to TLS_CRYPTO_INFO_READY scope and there will be no need for extra variables.

I don't see how to do that cleanly. I'd have to duplicate the copy_from_sockptr, which IMHO looks a lot worse. Is there another way?

Quoted 3 lines
Quoted 3 lines
net/tls/tls_main.c +0 -0
  	/* Ensure that TLS version and ciphers are same in both directions */
  	if (TLS_CRYPTO_INFO_READY(alt_crypto_info)) {
  		if (alt_crypto_info->version != crypto_info->version ||

[...]

Quoted 22 lines
Quoted 11 lines
net/tls/tls_sw.c +1 -6
@@ -2794,12 +2852,14 @@ int tls_set_sw_offload(struct sock *sk, int tx)
  	kfree(cctx->iv);
  	cctx->iv = NULL;
  free_priv:
-	if (tx) {
-		kfree(ctx->priv_ctx_tx);
-		ctx->priv_ctx_tx = NULL;
-	} else {
-		kfree(ctx->priv_ctx_rx);
-		ctx->priv_ctx_rx = NULL;
+	if (!new_crypto_info) {

net/tls/tls_sw.c int tls_set_sw_offload(struct sock *sk, int tx) · line 2865

Quoted 10 lines
net/tls/tls_sw.c +7 -0
+		if (tx) {
+			kfree(ctx->priv_ctx_tx);
+			ctx->priv_ctx_tx = NULL;
+		} else {
+			kfree(ctx->priv_ctx_rx);
+			ctx->priv_ctx_rx = NULL;
+		}
  	}
  out:
  	return rc;
Quoted 8 lines

I think we can avoid extra parameter and extra level of if{} constructions by checking if iv and rec_seq is already allocated and adjust init part the same way. I don't think we have to have separate error path because in case of any error during rekey procedure the connection becomes useless and application should indicate error to the other end. The code copies new crypto info to the current storage, so it assumes that all fields a properly filled and that means that this copy can be done earlier and use the same code path as first init code.

Rekey could fail because of memory allocation failures during crypto_aead_setkey. Userspace could choose to retry the key update, and we shouldn't necessarily kill off the connection in that case.

I think we need to keep the init/update distinction in the error paths for tls_set_sw_offload and do_tls_setsockopt_conf, otherwise we clear the crypto_info from the context and a new attempt to do the rekey will run through the full init path instead of the rekey path.

We could set crypto_info in tls_context before calling tls_set_sw_offload, but do_tls_setsockopt_conf would still have some differences since we need to validate that the version/cipher hasn't changed. I'll give that a try and see how much that improves things. It should reduce the churn a bit.

Thanks

Signature · 1 line
Sabrina
author maintainer
#9 lore raw

Re: [PATCH net-next 3/5] tls: implement rekey for TLS1.3

In reply to Apoorv Kothari
Quoted 62 lines · Apoorv Kothari
Quoted 9 lines
Quoted 5 lines · Kuniyuki Iwashima

Hi,

Thanks for posting this series!
We were working on the same feature.
CC Apoorv from s2n team.

Ah, cool. Does the behavior in those patches match what your implementation?

Thanks for submitting this, it looks great! We are working on testing this now.

Quoted 36 lines

[...]

Quoted 34 lines
Quoted 21 lines
net/tls/tls_main.c +3 -0
index fb1da1780f50..9be82aecd13e 100644
@@ -669,9 +669,12 @@ static int tls_getsockopt(struct sock *sk, int level, int optname,
 static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 				  unsigned int optlen, int tx)
 {
+	union tls_crypto_context tmp = {};
+	struct tls_crypto_info *old_crypto_info = NULL;
 	struct tls_crypto_info *crypto_info;
 	struct tls_crypto_info *alt_crypto_info;
 	struct tls_context *ctx = tls_get_ctx(sk);
+	bool update = false;
 	size_t optsize;
 	int rc = 0;
 	int conf;
@@ -687,9 +690,17 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
 		alt_crypto_info = &ctx->crypto_send.info;
 	}
 

net/tls/tls_main.c static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval, · line 699

Quoted 10 lines
net/tls/tls_main.c +7 -3
-	/* Currently we don't support set crypto info more than one time */
-	if (TLS_CRYPTO_INFO_READY(crypto_info))
-		return -EBUSY;
+	if (TLS_CRYPTO_INFO_READY(crypto_info)) {
+		/* Currently we only support setting crypto info more
+		 * than one time for TLS 1.3
+		 */
+		if (crypto_info->version != TLS_1_3_VERSION)
+			return -EBUSY;
+

Should we check this ?

Quoted 8 lines
Quoted 5 lines
                if (!tx && !key_update_pending)
                        return -EBUSY;

Otherwise we can set a new RX key even if the other end has not sent KeyUpdateRequest.

Maybe. My thinking was "let userspace shoot itself in the foot if it wants".

I feel avoiding foot-guns is probably the correct thing to do. The RFC also has a requirement that re-key(process messages with new key) should only happen after a KeyUpdate is received so it would be nice if the kTLS implemention can help enforce this.

Quoted 5 lines

Based on the RFC https://www.rfc-editor.org/rfc/rfc8446#section-4.6.3:
Additionally, both sides MUST enforce that a KeyUpdate with the old key is received before accepting any messages encrypted with the new key. Failure to do so may allow message truncation attacks.

Ok. I'll add that in v2, unless someone is strongly against it.

Thanks.

Signature · 1 line
Sabrina
author maintainer
#28 lore raw
In reply to Jakub Kicinski
Quoted 25 lines · Jakub Kicinski
Quoted 17 lines · On Wed, 18 Jan 2023 11:06:25 +0100 Sabrina Dubroca wrote:
Quoted 13 lines · Jakub Kicinski
Quoted 11 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 6 lines

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

Quoted 3 lines

IDK, we don't interpret any other content types/cmsgs, and for well behaved user space there should be no problem (right?). I'm weakly against, if nobody agrees with me you can keep as is.

I was concerned (I don't know if it's realistic) about a userspace application with two threads:

Thread A Thread B
-------- --------

read cmsg

read some data (still on the old key)

sets the new key

I guess one could claim that's a userspace bug.

František's implementation in gnutls doesn't seem to rely on this.

Apoorv, since you were also looking into key updates, do you have an opinion on pausing decryption/reads until userspace has provides the new key?

Quoted 20 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Quoted 6 lines
Quoted 3 lines

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

I'm pretty sure some devices support it. None of the vendors could be bothered to plumb in the kernel support, yet, tho.
I don't know of anyone supporting rekeying.

Quoted 7 lines
Quoted 2 lines

IIRC support for KeyUpdates is mandatory in TLS1.3, so currently the kernel can't claim to support 1.3, independent of offloading.

The problem is that we will not be able to rekey offloaded connections. For Tx it's a non-trivial problem given the current architecture. The offload is supposed to be transparent, we can't fail the rekey just because the TLS gotten offloaded.

What's their plan when the peer sends a KeyUpdate request then? Let the connection break?

Signature · 1 line
Sabrina
#29 lore raw
Quoted 44 lines
Quoted 25 lines · Jakub Kicinski
Quoted 17 lines · On Wed, 18 Jan 2023 11:06:25 +0100 Sabrina Dubroca wrote:
Quoted 13 lines · Jakub Kicinski
Quoted 11 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 6 lines

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

Quoted 3 lines

IDK, we don't interpret any other content types/cmsgs, and for well behaved user space there should be no problem (right?). I'm weakly against, if nobody agrees with me you can keep as is.

I was concerned (I don't know if it's realistic) about a userspace application with two threads:

Thread A Thread B
-------- --------

read cmsg

read some data (still on the old key)

sets the new key

Quoted 8 lines

I guess one could claim that's a userspace bug.

František's implementation in gnutls doesn't seem to rely on this.

Apoorv, since you were also looking into key updates, do you have an opinion on pausing decryption/reads until userspace has provides the new key?

There are a few reason I can think of why we would want the pausing behavior.

0) If possible, we should enforce correctness and prevent the userspace from doing something bad.

1) Pausing better aligns with the RFC since all subsequent messages should be encrypted with the new key.

From the RFC https://www.rfc-editor.org/rfc/rfc8446#section-4.6.3
After sending a KeyUpdate message, the sender SHALL send all its traffic using the next generation of keys, computed as described in Section 7.2. Upon receiving a KeyUpdate, the receiver MUST update its receiving keys.

2) Its possible to receive multiple KeyUpdate messages. If we allow the userspace to read more records, they would read multiple KeyUpdate messages. However since we currently track bool key_update_pending; using a bool, we would only require a single rekey and end up in a bad state. Its possible to fix this by not using bool but then the logic get more complicate and not worth it IMO.

Quoted 33 lines
Quoted 20 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Quoted 6 lines
Quoted 3 lines

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

I'm pretty sure some devices support it. None of the vendors could be bothered to plumb in the kernel support, yet, tho.
I don't know of anyone supporting rekeying.

Quoted 7 lines
Quoted 2 lines

IIRC support for KeyUpdates is mandatory in TLS1.3, so currently the kernel can't claim to support 1.3, independent of offloading.

The problem is that we will not be able to rekey offloaded connections. For Tx it's a non-trivial problem given the current architecture. The offload is supposed to be transparent, we can't fail the rekey just because the TLS gotten offloaded.

What's their plan when the peer sends a KeyUpdate request then? Let the connection break?

Signature · 1 line
Sabrina
#31 lore raw
In reply to Jakub Kicinski
Quoted 49 lines · Jakub Kicinski
Quoted 17 lines · On Wed, 18 Jan 2023 11:06:25 +0100 Sabrina Dubroca wrote:
Quoted 13 lines · Jakub Kicinski
Quoted 11 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 6 lines

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

IDK, we don't interpret any other content types/cmsgs, and for well behaved user space there should be no problem (right?). I'm weakly against, if nobody agrees with me you can keep as is.

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Quoted 6 lines
Quoted 3 lines

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

I'm pretty sure some devices support it. None of the vendors could be bothered to plumb in the kernel support, yet, tho.
I don't know of anyone supporting rekeying.

Quoted 7 lines
Quoted 2 lines

IIRC support for KeyUpdates is mandatory in TLS1.3, so currently the kernel can't claim to support 1.3, independent of offloading.

The problem is that we will not be able to rekey offloaded connections. For Tx it's a non-trivial problem given the current architecture. The offload is supposed to be transparent, we can't fail the rekey just because the TLS gotten offloaded.

But we really don't have any HW offload implementation for TLSv1.3. But it would be great to have SW implementation better align to RFC.

Quoted 2 lines
Quoted 2 lines

Some folks did tests with and without kTLS using nbdcopy and found a small but noticeable performance improvement (around 8-10%).

maintainer
#30 lore raw
In reply to Sabrina Dubroca
Quoted 10 lines · On Thu, 19 Jan 2023 16:40:39 +0100 Sabrina Dubroca wrote:
Quoted 7 lines
Quoted 2 lines

IIRC support for KeyUpdates is mandatory in TLS1.3, so currently the kernel can't claim to support 1.3, independent of offloading.

The problem is that we will not be able to rekey offloaded connections. For Tx it's a non-trivial problem given the current architecture. The offload is supposed to be transparent, we can't fail the rekey just because the TLS gotten offloaded.

What's their plan when the peer sends a KeyUpdate request then? Let the connection break?

I believe so, yes, just open a new connection. TLS rekeying seems to be extremely rare.

You mentioned nbd as a potential use case for kernel SW implementation. Can nbd rekey? Is use space responding to control messages in case of nbd?

#14 lore raw

PATCH 5/5 selftests: tls: add rekey tests

In reply to Sabrina Dubroca
Quoted 28 lines
---
 tools/testing/selftests/net/tls.c | 258 ++++++++++++++++++++++++++++++
 1 file changed, 258 insertions(+)
tools/testing/selftests/net/tls.c +15 -0
index 5f3adb28fee1..97c20e2246e1 100644
@@ -1453,6 +1453,264 @@ TEST_F(tls, shutdown_reuse)
     EXPECT_EQ(errno, EISCONN);
 }
 
+#define TLS_RECORD_TYPE_HANDSHAKE      0x16
+/* key_update, length 1, update_not_requested */
+static const char key_update_msg[] = "\x18\x00\x00\x01\x00";
+static void tls_send_keyupdate(struct __test_metadata *_metadata, int fd)
+{
+    size_t len = sizeof(key_update_msg);
+
+    EXPECT_EQ(tls_send_cmsg(fd, TLS_RECORD_TYPE_HANDSHAKE,
+                (char *)key_update_msg, len, 0),
+          len);
+}
+
+static void tls_recv_keyupdate(struct __test_metadata *_metadata, int fd, int flags)
+{
+    char buf[100];

tools/testing/selftests/net/tls.c TEST_F(tls, shutdown_reuse) · line 1480

Quoted 10 lines
tools/testing/selftests/net/tls.c +10 -0
+
+    EXPECT_EQ(tls_recv_cmsg(_metadata, fd, TLS_RECORD_TYPE_HANDSHAKE, buf, sizeof(buf), flags),
+          sizeof(key_update_msg));
+    EXPECT_EQ(memcmp(buf, key_update_msg, sizeof(key_update_msg)), 0);
+}
+
+/* set the key to 0 then 1 for RX, immediately to 1 for TX */
+TEST_F(tls_basic, rekey_rx)
+{
+    struct tls_crypto_info_keys tls12_0, tls12_1;

nit: Did you mean tls13_0 and tls13_1? There are also a few others in this patch.

Quoted 238 lines
tools/testing/selftests/net/tls.c +233 -0
+    char const *test_str = "test_message";
+    int send_len = strlen(test_str) + 1;
+    char buf[20];
+    int ret;
+
+    if (self->notls)
+        return;
+
+    tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128,
+                 &tls12_0, 0);
+    tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128,
+                 &tls12_1, 1);
+
+
+    ret = setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_1, tls12_1.len);
+    ASSERT_EQ(ret, 0);
+
+    ret = setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_0, tls12_0.len);
+    ASSERT_EQ(ret, 0);
+
+    ret = setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_1, tls12_1.len);
+    EXPECT_EQ(ret, 0);
+
+    EXPECT_EQ(send(self->fd, test_str, send_len, 0), send_len);
+    EXPECT_EQ(recv(self->cfd, buf, send_len, 0), send_len);
+    EXPECT_EQ(memcmp(buf, test_str, send_len), 0);
+}
+
+/* set the key to 0 then 1 for TX, immediately to 1 for RX */
+TEST_F(tls_basic, rekey_tx)
+{
+    struct tls_crypto_info_keys tls12_0, tls12_1;
+    char const *test_str = "test_message";
+    int send_len = strlen(test_str) + 1;
+    char buf[20];
+    int ret;
+
+    if (self->notls)
+        return;
+
+    tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128,
+                 &tls12_0, 0);
+    tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128,
+                 &tls12_1, 1);
+
+
+    ret = setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_0, tls12_0.len);
+    ASSERT_EQ(ret, 0);
+
+    ret = setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_1, tls12_1.len);
+    ASSERT_EQ(ret, 0);
+
+    ret = setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_1, tls12_1.len);
+    EXPECT_EQ(ret, 0);
+
+    EXPECT_EQ(send(self->fd, test_str, send_len, 0), send_len);
+    EXPECT_EQ(recv(self->cfd, buf, send_len, 0), send_len);
+    EXPECT_EQ(memcmp(buf, test_str, send_len), 0);
+}
+
+TEST_F(tls, rekey)
+{
+    char const *test_str_1 = "test_message_before_rekey";
+    char const *test_str_2 = "test_message_after_rekey";
+    struct tls_crypto_info_keys tls12;
+    int send_len;
+    char buf[100];
+
+    if (variant->tls_version != TLS_1_3_VERSION)
+        return;
+
+    /* initial send/recv */
+    send_len = strlen(test_str_1) + 1;
+    EXPECT_EQ(send(self->fd, test_str_1, send_len, 0), send_len);
+    EXPECT_EQ(recv(self->cfd, buf, send_len, 0), send_len);
+    EXPECT_EQ(memcmp(buf, test_str_1, send_len), 0);
+
+    /* update TX key */
+    tls_send_keyupdate(_metadata, self->fd);
+    tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 1);
+    EXPECT_EQ(setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12, tls12.len), 0);
+
+    /* send after rekey */
+    send_len = strlen(test_str_2) + 1;
+    EXPECT_EQ(send(self->fd, test_str_2, send_len, 0), send_len);
+
+    /* can't receive the KeyUpdate without a control message */
+    EXPECT_EQ(recv(self->cfd, buf, send_len, 0), -1);
+
+    /* get KeyUpdate */
+    tls_recv_keyupdate(_metadata, self->cfd, 0);
+
+    /* recv blocking -> -EKEYEXPIRED */
+    EXPECT_EQ(recv(self->cfd, buf, sizeof(buf), 0), -1);
+    EXPECT_EQ(errno, EKEYEXPIRED);
+
+    /* recv non-blocking -> -EKEYEXPIRED */
+    EXPECT_EQ(recv(self->cfd, buf, sizeof(buf), MSG_DONTWAIT), -1);
+    EXPECT_EQ(errno, EKEYEXPIRED);
+
+    /* update RX key */
+    EXPECT_EQ(setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12, tls12.len), 0);
+
+    /* recv after rekey */
+    EXPECT_NE(recv(self->cfd, buf, send_len, 0), -1);
+    EXPECT_EQ(memcmp(buf, test_str_2, send_len), 0);
+}
+
+TEST_F(tls, rekey_peek)
+{
+    char const *test_str_1 = "test_message_before_rekey";
+    struct tls_crypto_info_keys tls12;
+    int send_len;
+    char buf[100];
+
+    if (variant->tls_version != TLS_1_3_VERSION)
+        return;
+
+    send_len = strlen(test_str_1) + 1;
+    EXPECT_EQ(send(self->fd, test_str_1, send_len, 0), send_len);
+
+    /* update TX key */
+    tls_send_keyupdate(_metadata, self->fd);
+    tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 1);
+    EXPECT_EQ(setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12, tls12.len), 0);
+
+    EXPECT_EQ(recv(self->cfd, buf, sizeof(buf), MSG_PEEK), send_len);
+    EXPECT_EQ(memcmp(buf, test_str_1, send_len), 0);
+
+    EXPECT_EQ(recv(self->cfd, buf, send_len, 0), send_len);
+    EXPECT_EQ(memcmp(buf, test_str_1, send_len), 0);
+
+    /* can't receive the KeyUpdate without a control message */
+    EXPECT_EQ(recv(self->cfd, buf, send_len, MSG_PEEK), -1);
+
+    /* peek KeyUpdate */
+    tls_recv_keyupdate(_metadata, self->cfd, MSG_PEEK);
+
+    /* get KeyUpdate */
+    tls_recv_keyupdate(_metadata, self->cfd, 0);
+
+    /* update RX key */
+    EXPECT_EQ(setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12, tls12.len), 0);
+}
+
+TEST_F(tls, splice_rekey)
+{
+    int send_len = TLS_PAYLOAD_MAX_LEN / 2;
+    char mem_send[TLS_PAYLOAD_MAX_LEN];
+    char mem_recv[TLS_PAYLOAD_MAX_LEN];
+    struct tls_crypto_info_keys tls12;
+    int p[2];
+
+    if (variant->tls_version != TLS_1_3_VERSION)
+        return;
+
+    memrnd(mem_send, sizeof(mem_send));
+
+    ASSERT_GE(pipe(p), 0);
+    EXPECT_EQ(send(self->fd, mem_send, send_len, 0), send_len);
+
+    /* update TX key */
+    tls_send_keyupdate(_metadata, self->fd);
+    tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 1);
+    EXPECT_EQ(setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12, tls12.len), 0);
+
+    EXPECT_EQ(send(self->fd, mem_send, send_len, 0), send_len);
+
+    EXPECT_EQ(splice(self->cfd, NULL, p[1], NULL, TLS_PAYLOAD_MAX_LEN, 0), send_len);
+    EXPECT_EQ(read(p[0], mem_recv, send_len), send_len);
+    EXPECT_EQ(memcmp(mem_send, mem_recv, send_len), 0);
+
+    /* can't splice the KeyUpdate */
+    EXPECT_EQ(splice(self->cfd, NULL, p[1], NULL, TLS_PAYLOAD_MAX_LEN, 0), -1);
+    EXPECT_EQ(errno, EINVAL);
+
+    /* peek KeyUpdate */
+    tls_recv_keyupdate(_metadata, self->cfd, MSG_PEEK);
+
+    /* get KeyUpdate */
+    tls_recv_keyupdate(_metadata, self->cfd, 0);
+
+    /* can't splice before updating the key */
+    EXPECT_EQ(splice(self->cfd, NULL, p[1], NULL, TLS_PAYLOAD_MAX_LEN, 0), -1);
+    EXPECT_EQ(errno, EKEYEXPIRED);
+
+    /* update RX key */
+    EXPECT_EQ(setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12, tls12.len), 0);
+
+    EXPECT_EQ(splice(self->cfd, NULL, p[1], NULL, TLS_PAYLOAD_MAX_LEN, 0), send_len);
+    EXPECT_EQ(read(p[0], mem_recv, send_len), send_len);
+    EXPECT_EQ(memcmp(mem_send, mem_recv, send_len), 0);
+}
+
+TEST_F(tls, rekey_getsockopt)
+{
+    struct tls_crypto_info_keys tls12;
+    struct tls_crypto_info_keys tls12_get;
+    socklen_t len;
+
+    tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 0);
+
+    len = tls12.len;
+    EXPECT_EQ(getsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_get, &len), 0);
+    EXPECT_EQ(len, tls12.len);
+    EXPECT_EQ(memcmp(&tls12_get, &tls12, tls12.len), 0);
+
+    len = tls12.len;
+    EXPECT_EQ(getsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_get, &len), 0);
+    EXPECT_EQ(len, tls12.len);
+    EXPECT_EQ(memcmp(&tls12_get, &tls12, tls12.len), 0);
+
+    if (variant->tls_version != TLS_1_3_VERSION)
+        return;
+
+    tls_send_keyupdate(_metadata, self->fd);
+    tls_crypto_info_init(variant->tls_version, variant->cipher_type, &tls12, 1);
+    EXPECT_EQ(setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12, tls12.len), 0);
+
+    tls_recv_keyupdate(_metadata, self->cfd, 0);
+    EXPECT_EQ(setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12, tls12.len), 0);
+
+    len = tls12.len;
+    EXPECT_EQ(getsockopt(self->fd, SOL_TLS, TLS_TX, &tls12_get, &len), 0);
+    EXPECT_EQ(len, tls12.len);
+    EXPECT_EQ(memcmp(&tls12_get, &tls12, tls12.len), 0);
+
+    len = tls12.len;
+    EXPECT_EQ(getsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12_get, &len), 0);
+    EXPECT_EQ(len, tls12.len);
+    EXPECT_EQ(memcmp(&tls12_get, &tls12, tls12.len), 0);
+}
+
 FIXTURE(tls_err)
 {
     int fd, cfd;
Signature · 1 line
2.38.1
#19 lore raw
In reply to Gal Pressman
Quoted 46 lines · Gal Pressman
Quoted 45 lines · Jakub Kicinski
Quoted 17 lines · On Wed, 18 Jan 2023 11:06:25 +0100 Sabrina Dubroca wrote:
Quoted 13 lines · Jakub Kicinski
Quoted 11 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 6 lines

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

IDK, we don't interpret any other content types/cmsgs, and for well behaved user space there should be no problem (right?). I'm weakly against, if nobody agrees with me you can keep as is.

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

Quoted 8 lines
Quoted 4 lines

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

Quoted 8 lines
Quoted 2 lines

I'm pretty sure some devices support it. None of the vendors could be bothered to plumb in the kernel support, yet, tho.

Our device supports TLS 1.3, it's in our plans to add driver/kernel support.

Quoted 1 line

I don't know of anyone supporting rekeying.

Boris, Tariq, do you know?

Rekeying is not trivial to get right with offload. There are at least two problems to solve:
1. On transmit, we need to handle both the new and the old key for new and old (retransmitted) data, respectively. Our device will be able to hold both keys in parallel and to choose the right one at the cost of an if statement in the data-path. Alternatively, we can just fallback to software for the old key and focus on the new key.
2. On Rx, packets with the new key may arrive before the key is installed unless we design a mechanism for preemptively setting the next key in HW. As a result, we may get a resync on every rekey.

Have you considered an API to preemptively set the next key in the kernel such that there is never a need to stop the datapath? I think that the change in SSL libraries is minor and it can really help KTLS.

author maintainer
#20 lore raw

(adding Simo and Daiki for the OpenSSL/GnuTLS sides)

Quoted 56 lines · Boris Pismenny
Quoted 46 lines · Gal Pressman
Quoted 45 lines · Jakub Kicinski
Quoted 17 lines · On Wed, 18 Jan 2023 11:06:25 +0100 Sabrina Dubroca wrote:
Quoted 13 lines · Jakub Kicinski
Quoted 11 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 6 lines

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

IDK, we don't interpret any other content types/cmsgs, and for well behaved user space there should be no problem (right?). I'm weakly against, if nobody agrees with me you can keep as is.

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

Quoted 8 lines
Quoted 4 lines

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

Quoted 8 lines
Quoted 2 lines

I'm pretty sure some devices support it. None of the vendors could be bothered to plumb in the kernel support, yet, tho.

Our device supports TLS 1.3, it's in our plans to add driver/kernel support.

Quoted 1 line

I don't know of anyone supporting rekeying.

Boris, Tariq, do you know?

Quoted 7 lines

Rekeying is not trivial to get right with offload. There are at least two problems to solve:
1. On transmit, we need to handle both the new and the old key for new and old (retransmitted) data, respectively. Our device will be able to hold both keys in parallel and to choose the right one at the cost of an if statement in the data-path. Alternatively, we can just fallback to software for the old key and focus on the new key.

We'll need to keep the old key around until we know all the records using it have been fully received, right? And that could be multiple old keys, in case of a quick series of key updates.

Quoted 7 lines

2. On Rx, packets with the new key may arrive before the key is installed unless we design a mechanism for preemptively setting the next key in HW. As a result, we may get a resync on every rekey.

Have you considered an API to preemptively set the next key in the kernel such that there is never a need to stop the datapath? I think that the change in SSL libraries is minor and it can really help KTLS.

I don't like the idea of having some unused key stored in the kernel for long durations too much.

You can't be sure that there will be only one rekey in the next short interval, so you'll need to be able to handle those resyncs anyway, in case userspace is too slow providing the 3rd key (for the 2nd rekey). For example, the RFC mentions:

If implementations independently send their own KeyUpdates with request_update set to "update_requested" and they cross in flight, then each side will also send a response, with the result that each side increments by two generations.

https://www.rfc-editor.org/rfc/rfc8446#section-4.6.3

So I think what you're suggesting can only be an optimization, not a full solution.

I hope I'm not getting too confused by all this.

Signature · 1 line
Sabrina
#21 lore raw
Quoted 65 lines
Quoted 56 lines · Boris Pismenny
Quoted 46 lines · Gal Pressman
Quoted 45 lines · Jakub Kicinski
Quoted 17 lines · On Wed, 18 Jan 2023 11:06:25 +0100 Sabrina Dubroca wrote:
Quoted 13 lines · Jakub Kicinski
Quoted 11 lines · On Tue, 17 Jan 2023 14:45:26 +0100 Sabrina Dubroca wrote:

This adds support for receiving KeyUpdate messages (RFC 8446, 4.6.3 [1]). A sender transmits a KeyUpdate message and then changes its TX key. The receiver should react by updating its RX key before processing the next message.

This patchset implements key updates by:
1. pausing decryption when a KeyUpdate message is received, to avoid attempting to use the old key to decrypt a record encrypted with the new key
2. returning -EKEYEXPIRED to syscalls that cannot receive the KeyUpdate message, until the rekey has been performed by userspace

Quoted 2 lines

Why? We return to user space after hitting a cmsg, don't we? If the user space wants to keep reading with the old key - 🤷️

Quoted 6 lines

But they won't be able to read anything. Either we don't pause decryption, and the socket is just broken when we look at the next record, or we pause, and there's nothing to read until the rekey is done. I think that -EKEYEXPIRED is better than breaking the socket just because a read snuck in between getting the cmsg and setting the new key.

IDK, we don't interpret any other content types/cmsgs, and for well behaved user space there should be no problem (right?). I'm weakly against, if nobody agrees with me you can keep as is.

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines

3. passing the KeyUpdate message to userspace as a control message
4. allowing updates of the crypto_info via the TLS_TX/TLS_RX setsockopts

Quoted 8 lines
Quoted 4 lines

This API has been tested with gnutls to make sure that it allows userspace libraries to implement key updates [2]. Thanks to Frantisek Krenzelok <[email protected]> for providing the implementation in gnutls and testing the kernel patches.

Please explain why - the kernel TLS is not faster than user space, the point of it is primarily to enable offload. And you don't add offload support here.

Well, TLS1.3 support was added 4 years ago, and yet the offload still doesn't support 1.3 at all.

Quoted 8 lines
Quoted 2 lines

I'm pretty sure some devices support it. None of the vendors could be bothered to plumb in the kernel support, yet, tho.

Our device supports TLS 1.3, it's in our plans to add driver/kernel support.

Quoted 1 line

I don't know of anyone supporting rekeying.

Boris, Tariq, do you know?

Quoted 7 lines

Rekeying is not trivial to get right with offload. There are at least two problems to solve:
1. On transmit, we need to handle both the new and the old key for new and old (retransmitted) data, respectively. Our device will be able to hold both keys in parallel and to choose the right one at the cost of an if statement in the data-path. Alternatively, we can just fallback to software for the old key and focus on the new key.

Quoted 3 lines

We'll need to keep the old key around until we know all the records using it have been fully received, right? And that could be multiple old keys, in case of a quick series of key updates.

Why does the hardware implementation need to store old keys? Does the need for retransmitted data assume we are operating in TLS_HW_RECORD mode and the hardware is also implementing the TCP stack?

The TLS RFC assumes that the underlying transport layer provides reliable and in-order deliver so storing previous keys and encrypting 'old' data would be quite divergent from normal TLS behavior. Is the TLS_HW_RECORD mode processing TLS records out of order? If the hardware offload is handling the TCP networking stack then I feel it should also handle the retransmission of lost data.

https://www.rfc-editor.org/rfc/rfc8446#section-1
the only requirement from the underlying
transport is a reliable, in-order data stream.

Quoted 27 lines
Quoted 7 lines

2. On Rx, packets with the new key may arrive before the key is installed unless we design a mechanism for preemptively setting the next key in HW. As a result, we may get a resync on every rekey.

Have you considered an API to preemptively set the next key in the kernel such that there is never a need to stop the datapath? I think that the change in SSL libraries is minor and it can really help KTLS.

I don't like the idea of having some unused key stored in the kernel for long durations too much.

You can't be sure that there will be only one rekey in the next short interval, so you'll need to be able to handle those resyncs anyway, in case userspace is too slow providing the 3rd key (for the 2nd rekey). For example, the RFC mentions:

If implementations independently send their own KeyUpdates with request_update set to "update_requested" and they cross in flight, then each side will also send a response, with the result that each side increments by two generations.

   https://www.rfc-editor.org/rfc/rfc8446#section-4.6.3

So I think what you're suggesting can only be an optimization, not a full solution.

I hope I'm not getting too confused by all this.

Signature · 1 line
Apoorv
maintainer
#22 lore raw
Quoted 7 lines · On Wed, 25 Jan 2023 10:47:20 -0800 Apoorv Kothari wrote:
Quoted 3 lines

We'll need to keep the old key around until we know all the records using it have been fully received, right? And that could be multiple old keys, in case of a quick series of key updates.

Why does the hardware implementation need to store old keys? Does the need for retransmitted data assume we are operating in TLS_HW_RECORD mode and the hardware is also implementing the TCP stack?

We're talking about the Tx direction, the packets are queued to the lower layers of the stack unencrypted, and get encrypted by the NIC. Until TCP gets acks for all the data awaiting offloaded crypto - we must hold onto the keys.

Rx direction is much simpler indeed.

Quoted 6 lines

The TLS RFC assumes that the underlying transport layer provides reliable and in-order deliver so storing previous keys and encrypting 'old' data would be quite divergent from normal TLS behavior. Is the TLS_HW_RECORD mode processing TLS records out of order? If the hardware offload is handling the TCP networking stack then I feel it should also handle the retransmission of lost data.

Ignore TLS_HW_RECORD, it's a ToE offload, the offload we care about just offloads encryption.

#23 lore raw
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Wed, 25 Jan 2023 10:47:20 -0800 Apoorv Kothari wrote:
Quoted 3 lines

We'll need to keep the old key around until we know all the records using it have been fully received, right? And that could be multiple old keys, in case of a quick series of key updates.

Quoted 8 lines
Quoted 3 lines

Why does the hardware implementation need to store old keys? Does the need for retransmitted data assume we are operating in TLS_HW_RECORD mode and the hardware is also implementing the TCP stack?

We're talking about the Tx direction, the packets are queued to the lower layers of the stack unencrypted, and get encrypted by the NIC. Until TCP gets acks for all the data awaiting offloaded crypto - we must hold onto the keys.

Is this because the NIC does not cache the already encrypted outgoing packets?

If that is the case is it _guaranteed_ that the re-encrypted packets are exactly identical to the previously sent ones?

If it is not guaranteed, are you blocking use of AES GCM and any other block cipher that may have very bad failure modes in a situation like this (in the case of AES GCM I am thinking of IV reuse) ?

Quoted 13 lines

Rx direction is much simpler indeed.

Quoted 6 lines

The TLS RFC assumes that the underlying transport layer provides reliable and in-order deliver so storing previous keys and encrypting 'old' data would be quite divergent from normal TLS behavior. Is the TLS_HW_RECORD mode processing TLS records out of order? If the hardware offload is handling the TCP networking stack then I feel it should also handle the retransmission of lost data.

Ignore TLS_HW_RECORD, it's a ToE offload, the offload we care about just offloads encryption.

Signature · 3 lines
Simo Sorce
RHEL Crypto Team
Red Hat, Inc
maintainer
#24 lore raw
Quoted 7 lines · On Wed, 25 Jan 2023 16:17:26 -0500 Simo Sorce wrote:
Quoted 4 lines

We're talking about the Tx direction, the packets are queued to the lower layers of the stack unencrypted, and get encrypted by the NIC. Until TCP gets acks for all the data awaiting offloaded crypto - we must hold onto the keys.

Is this because the NIC does not cache the already encrypted outgoing packets?

NIC can't cache outgoing packets, there's too many and NIC is supposed to only do crypto. TCP stack is responsible for handing rtx.

Quoted 2 lines

If that is the case is it _guaranteed_ that the re-encrypted packets are exactly identical to the previously sent ones?

In terms of payload, yes. Modulo zero-copy cases we don't need to get into.

Quoted 3 lines

If it is not guaranteed, are you blocking use of AES GCM and any other block cipher that may have very bad failure modes in a situation like this (in the case of AES GCM I am thinking of IV reuse) ?

I don't know what you mean.

#25 lore raw
Quoted 15 lines · Jakub Kicinski
Quoted 7 lines · On Wed, 25 Jan 2023 16:17:26 -0500 Simo Sorce wrote:
Quoted 4 lines

We're talking about the Tx direction, the packets are queued to the lower layers of the stack unencrypted, and get encrypted by the NIC. Until TCP gets acks for all the data awaiting offloaded crypto - we must hold onto the keys.

Is this because the NIC does not cache the already encrypted outgoing packets?

NIC can't cache outgoing packets, there's too many and NIC is supposed to only do crypto. TCP stack is responsible for handing rtx.

Quoted 2 lines

If that is the case is it _guaranteed_ that the re-encrypted packets are exactly identical to the previously sent ones?

Quoted 8 lines

In terms of payload, yes. Modulo zero-copy cases we don't need to get into.

Quoted 3 lines

If it is not guaranteed, are you blocking use of AES GCM and any other block cipher that may have very bad failure modes in a situation like this (in the case of AES GCM I am thinking of IV reuse) ?

I don't know what you mean.

The question was if there is *any* case where re-transmission can cause different data to be encrypted with the same key + same IV

Simo.

Signature · 3 lines
Simo Sorce
RHEL Crypto Team
Red Hat, Inc
maintainer
#26 lore raw
Quoted 8 lines · On Wed, 25 Jan 2023 18:05:38 -0500 Simo Sorce wrote:
Quoted 5 lines
Quoted 3 lines

If it is not guaranteed, are you blocking use of AES GCM and any other block cipher that may have very bad failure modes in a situation like this (in the case of AES GCM I am thinking of IV reuse) ?

I don't know what you mean.

The question was if there is *any* case where re-transmission can cause different data to be encrypted with the same key + same IV

Not in valid use cases. With zero-copy / sendfile Tx technically the page from the page cache can change between tx and rtx, but the user needs to opt in explicitly acknowledging the application will prevent this from happening. If they don't opt-in we'll copy the data.

#27 lore raw
Quoted 7 lines · Jakub Kicinski
Quoted 6 lines · On Wed, 25 Jan 2023 18:05:38 -0500 Simo Sorce wrote:
Quoted 5 lines
Quoted 3 lines

If it is not guaranteed, are you blocking use of AES GCM and any other block cipher that may have very bad failure modes in a situation like this (in the case of AES GCM I am thinking of IV reuse) ?

I don't know what you mean.

Quoted 8 lines
Quoted 2 lines

The question was if there is *any* case where re-transmission can cause different data to be encrypted with the same key + same IV

Not in valid use cases. With zero-copy / sendfile Tx technically the page from the page cache can change between tx and rtx, but the user needs to opt in explicitly acknowledging the application will prevent this from happening. If they don't opt-in we'll copy the data.

Uhmm is there a way to detect this happening and abort further crypto operations in case it happens ?

Simo.

Signature · 3 lines
Simo Sorce
RHEL Crypto Team
Red Hat, Inc