Skip to contentSkip to conversation

Fix use-after-free after the TLS device goes down and up

PATCH Applied Maxim Mikityanskiy 3 participants 5 messages Jun 1, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover Fix use-after-free after the TLS device goes down and up

This small series fixes a use-after-free bug in the TLS offload code. The first patch is a preparation for the second one, and the second is the fix itself.

v2 changes:

Remove unneeded EXPORT_SYMBOL_GPL.

Maxim Mikityanskiy (2):
net/tls: Replace TLS_RX_SYNC_RUNNING with RCU
net/tls: Fix use-after-free after the TLS device goes down and up

 include/net/tls.h             | 10 +++++-
 net/tls/tls_device.c          | 60 ++++++++++++++++++++++++++++-------
 net/tls/tls_device_fallback.c |  7 ++++
 net/tls/tls_main.c            |  1 +
 4 files changed, 66 insertions(+), 12 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH v2 1/2 net/tls: Replace TLS_RX_SYNC_RUNNING with RCU

RCU synchronization is guaranteed to finish in finite time, unlike a busy loop that polls a flag. This patch is a preparation for the bugfix in the next patch, where the same synchronize_net() call will also be used to sync with the TX datapath.

---
 include/net/tls.h    |  1 -
 net/tls/tls_device.c | 10 +++-------
 2 files changed, 3 insertions(+), 8 deletions(-)
include/net/tls.h +0 -1
index 3eccb525e8f7..6531ace2a68b 100644
@@ -193,7 +193,6 @@ struct tls_offload_context_tx {
 	(sizeof(struct tls_offload_context_tx) + TLS_DRIVER_STATE_SIZE_TX)
 
 enum tls_context_flags {
-	TLS_RX_SYNC_RUNNING = 0,
 	/* Unlike RX where resync is driven entirely by the core in TX only
 	 * the driver knows when things went out of sync, so we need the flag
 	 * to be atomic.
net/tls/tls_device.c +3 -7
index 76a6f8c2eec4..171752cd6910 100644
@@ -680,15 +680,13 @@ static void tls_device_resync_rx(struct tls_context *tls_ctx,
 	struct tls_offload_context_rx *rx_ctx = tls_offload_ctx_rx(tls_ctx);
 	struct net_device *netdev;
 
-	if (WARN_ON(test_and_set_bit(TLS_RX_SYNC_RUNNING, &tls_ctx->flags)))
-		return;
-
 	trace_tls_device_rx_resync_send(sk, seq, rcd_sn, rx_ctx->resync_type);
+	rcu_read_lock();
 	netdev = READ_ONCE(tls_ctx->netdev);
 	if (netdev)
 		netdev->tlsdev_ops->tls_dev_resync(netdev, sk, seq, rcd_sn,
 						   TLS_OFFLOAD_CTX_DIR_RX);
-	clear_bit_unlock(TLS_RX_SYNC_RUNNING, &tls_ctx->flags);
+	rcu_read_unlock();
 	TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXDEVICERESYNC);
 }
 
@@ -1300,9 +1298,7 @@ static int tls_device_down(struct net_device *netdev)
 			netdev->tlsdev_ops->tls_dev_del(netdev, ctx,
 							TLS_OFFLOAD_CTX_DIR_RX);
 		WRITE_ONCE(ctx->netdev, NULL);
-		smp_mb__before_atomic(); /* pairs with test_and_set_bit() */
-		while (test_bit(TLS_RX_SYNC_RUNNING, &ctx->flags))
-			usleep_range(10, 200);
+		synchronize_net();
 		dev_put(netdev);
 		list_del_init(&ctx->list);
 
Signature · 1 line
2.25.1
#2 lore raw

PATCH v2 2/2 net/tls: Fix use-after-free after the TLS device goes down and up

In reply to Maxim Mikityanskiy

When a netdev with active TLS offload goes down, tls_device_down is called to stop the offload and tear down the TLS context. However, the socket stays alive, and it still points to the TLS context, which is now deallocated. If a netdev goes up, while the connection is still active, and the data flow resumes after a number of TCP retransmissions, it will lead to a use-after-free of the TLS context.

This commit addresses this bug by keeping the context alive until its normal destruction, and implements the necessary fallbacks, so that the connection can resume in software (non-offloaded) kTLS mode.

On the TX side tls_sw_fallback is used to encrypt all packets. The RX side already has all the necessary fallbacks, because receiving non-decrypted packets is supported. The thing needed on the RX side is to block resync requests, which are normally produced after receiving non-decrypted packets.

The necessary synchronization is implemented for a graceful teardown:
first the fallbacks are deployed, then the driver resources are released (it used to be possible to have a tls_dev_resync after tls_dev_del).

A new flag called TLS_RX_DEV_DEGRADED is added to indicate the fallback mode. It's used to skip the RX resync logic completely, as it becomes useless, and some objects may be released (for example, resync_async, which is allocated and freed by the driver).

Fixes e8f69799810c ("net/tls: Add generic NIC offload infrastructure")
---
 include/net/tls.h             |  9 ++++++
 net/tls/tls_device.c          | 52 +++++++++++++++++++++++++++++++----
 net/tls/tls_device_fallback.c |  7 +++++
 net/tls/tls_main.c            |  1 +
 4 files changed, 64 insertions(+), 5 deletions(-)
include/net/tls.h +9 -0
index 6531ace2a68b..8341a8d1e807 100644
@@ -193,6 +193,11 @@ struct tls_offload_context_tx {
 	(sizeof(struct tls_offload_context_tx) + TLS_DRIVER_STATE_SIZE_TX)
 
 enum tls_context_flags {
+	/* tls_device_down was called after the netdev went down, device state
+	 * was released, and kTLS works in software, even though rx_conf is
+	 * still TLS_HW (needed for transition).
+	 */
+	TLS_RX_DEV_DEGRADED = 0,
 	/* Unlike RX where resync is driven entirely by the core in TX only
 	 * the driver knows when things went out of sync, so we need the flag
 	 * to be atomic.
@@ -265,6 +270,7 @@ struct tls_context {
 
 	/* cache cold stuff */
 	struct proto *sk_proto;
+	struct sock *sk;
 
 	void (*sk_destruct)(struct sock *sk);
 
@@ -447,6 +453,9 @@ static inline u16 tls_user_config(struct tls_context *ctx, bool tx)
 struct sk_buff *
 tls_validate_xmit_skb(struct sock *sk, struct net_device *dev,
 		      struct sk_buff *skb);
+struct sk_buff *
+tls_validate_xmit_skb_sw(struct sock *sk, struct net_device *dev,
+			 struct sk_buff *skb);
 
 static inline bool tls_is_sk_tx_device_offloaded(struct sock *sk)
 {
net/tls/tls_device.c +47 -5

84 lines not on this page · raw message

net/tls/tls_device_fallback.c +7 -0
index cacf040872c7..e40bedd112b6 100644
@@ -431,6 +431,13 @@ struct sk_buff *tls_validate_xmit_skb(struct sock *sk,
 }
 EXPORT_SYMBOL_GPL(tls_validate_xmit_skb);
 
+struct sk_buff *tls_validate_xmit_skb_sw(struct sock *sk,
+					 struct net_device *dev,
+					 struct sk_buff *skb)
+{
+	return tls_sw_fallback(sk, skb);
+}
+
 struct sk_buff *tls_encrypt_skb(struct sk_buff *skb)
 {
 	return tls_sw_fallback(skb->sk, skb);
net/tls/tls_main.c +1 -0
index 47b7c5334c34..fde56ff49163 100644
@@ -636,6 +636,7 @@ struct tls_context *tls_ctx_create(struct sock *sk)
 	mutex_init(&ctx->tx_lock);
 	rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
 	ctx->sk_proto = READ_ONCE(sk->sk_prot);
+	ctx->sk = sk;
 	return ctx;
 }
 
Signature · 1 line
2.25.1
#4 lore raw
In reply to Maxim Mikityanskiy
This series was applied to netdev/net.git (refs/heads/master):

Hello:

This series was applied to netdev/net.git (refs/heads/master):

Quoted 9 lines · On Tue, 1 Jun 2021 15:07:58 +0300 you wrote:

This small series fixes a use-after-free bug in the TLS offload code. The first patch is a preparation for the second one, and the second is the fix itself.

v2 changes:

Remove unneeded EXPORT_SYMBOL_GPL.

[...]

Here is the summary with links:
- [net,v2,1/2] net/tls: Replace TLS_RX_SYNC_RUNNING with RCU

    https://git.kernel.org/netdev/net/c/05fc8b6cbd4f
  - [net,v2,2/2] net/tls: Fix use-after-free after the TLS device goes down and up
    https://git.kernel.org/netdev/net/c/c55dcdd435aa

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html