Skip to contentSkip to conversation

tcp: tcp challenge ack fixes

PATCH Applied Eric Dumazet 3 participants 6 messages Aug 30, 2022 – Sep 1, 2022

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover tcp: tcp challenge ack fixes

From: Eric Dumazet <[email protected]>

syzbot found a typical data-race addressed in the first patch.

While we are at it, second patch makes the global rate limit per net-ns and disabled by default.

Eric Dumazet (2):
tcp: annotate data-race around challenge_timestamp
tcp: make global challenge ack rate limitation per net-ns and default
disabled

 Documentation/networking/ip-sysctl.rst |  5 ++++-
 include/net/netns/ipv4.h               |  2 ++
 net/ipv4/tcp_input.c                   | 21 +++++++++++----------
 net/ipv4/tcp_ipv4.c                    |  6 ++++--
 4 files changed, 21 insertions(+), 13 deletions(-)
Signature · 1 line
2.37.2.672.g94769d06f0-goog
#1 lore raw

PATCH 1/2 tcp: annotate data-race around challenge_timestamp

From: Eric Dumazet <[email protected]>

challenge_timestamp can be read an written by concurrent threads.

This was expected, but we need to annotate the race to avoid potential issues.

Following patch moves challenge_timestamp and challenge_count to per-netns storage to provide better isolation.

Fixes 354e4aa391ed ("tcp: RFC 5961 5.2 Blind Data Injection Attack Mitigation")
---
 net/ipv4/tcp_input.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/ipv4/tcp_input.c +2 -2
index ab5f0ea166f1a0535e299a9051406b5e2895f1f0..c184e15397a28ccfbac142ff0f1d05d555623147 100644
@@ -3629,11 +3629,11 @@ static void tcp_send_challenge_ack(struct sock *sk)
 
 	/* Then check host-wide RFC 5961 rate limit. */
 	now = jiffies / HZ;
-	if (now != challenge_timestamp) {
+	if (now != READ_ONCE(challenge_timestamp)) {
 		u32 ack_limit = READ_ONCE(net->ipv4.sysctl_tcp_challenge_ack_limit);
 		u32 half = (ack_limit + 1) >> 1;
 
-		challenge_timestamp = now;
+		WRITE_ONCE(challenge_timestamp, now);
 		WRITE_ONCE(challenge_count, half + prandom_u32_max(ack_limit));
 	}
 	count = READ_ONCE(challenge_count);
Signature · 1 line
2.37.2.672.g94769d06f0-goog
maintainer Acked-by on v1 1/2
#2 lore raw

Re: [PATCH net 1/2] tcp: annotate data-race around challenge_timestamp

Quoted 7 lines · Eric Dumazet

From: Eric Dumazet <[email protected]>

challenge_timestamp can be read an written by concurrent threads.

This was expected, but we need to annotate the race to avoid potential issues.

Quoted 7 lines

Following patch moves challenge_timestamp and challenge_count to per-netns storage to provide better isolation.

Fixes 354e4aa391ed ("tcp: RFC 5961 5.2 Blind Data Injection Attack Mitigation")

---

Thanks, Eric!

neal

#3 lore raw

PATCH 2/2 tcp: make global challenge ack rate limitation per net-ns and default disabled

In reply to Eric Dumazet

From: Eric Dumazet <[email protected]>

Because per host rate limiting has been proven problematic (side channel attacks can be based on it), per host rate limiting of challenge acks ideally should be per netns and turned off by default.

This is a long due followup of following commits:

083ae308280d ("tcp: enable per-socket rate limiting of all 'challenge acks'") f2b2c582e824 ("tcp: mitigate ACK loops for connections as tcp_sock") 75ff39ccc1bd ("tcp: make challenge acks less predictable")

Cc Jason Baron <[email protected]>
Cc Neal Cardwell <[email protected]>
---
 Documentation/networking/ip-sysctl.rst |  5 ++++-
 include/net/netns/ipv4.h               |  2 ++
 net/ipv4/tcp_input.c                   | 21 +++++++++++----------
 net/ipv4/tcp_ipv4.c                    |  6 ++++--
 4 files changed, 21 insertions(+), 13 deletions(-)
Documentation/networking/ip-sysctl.rst +4 -1
index 56cd4ea059b2a33cde01d8db598b160762db1478..a759872a2883bbce925833007a540f06c68a9442 100644
@@ -1035,7 +1035,10 @@ tcp_limit_output_bytes - INTEGER
 tcp_challenge_ack_limit - INTEGER
 	Limits number of Challenge ACK sent per second, as recommended
 	in RFC 5961 (Improving TCP's Robustness to Blind In-Window Attacks)
-	Default: 1000
+	Note that this per netns rate limit can allow some side channel
+	attacks and probably should not be enabled.
+	TCP stack implements per TCP socket limits anyway.
+	Default: INT_MAX (unlimited)
 
 UDP variables
 =============
include/net/netns/ipv4.h +2 -0
index c7320ef356d940aefba708ea98de502253d2b04d..6320a76cefdcdf6232b7a94ddd0c4a433084887d 100644
@@ -179,6 +179,8 @@ struct netns_ipv4 {
 	unsigned int sysctl_tcp_fastopen_blackhole_timeout;
 	atomic_t tfo_active_disable_times;
 	unsigned long tfo_active_disable_stamp;
+	u32 tcp_challenge_timestamp;
+	u32 tcp_challenge_count;
 
 	int sysctl_udp_wmem_min;
 	int sysctl_udp_rmem_min;
net/ipv4/tcp_input.c +11 -10
index c184e15397a28ccfbac142ff0f1d05d555623147..b85a9f755da41505b35bc64e1fa8995581660e90 100644
@@ -3614,12 +3614,9 @@ bool tcp_oow_rate_limited(struct net *net, const struct sk_buff *skb,
 /* RFC 5961 7 [ACK Throttling] */
 static void tcp_send_challenge_ack(struct sock *sk)
 {
-	/* unprotected vars, we dont care of overwrites */
-	static u32 challenge_timestamp;
-	static unsigned int challenge_count;
 	struct tcp_sock *tp = tcp_sk(sk);
 	struct net *net = sock_net(sk);
-	u32 count, now;
+	u32 count, now, ack_limit;
 
 	/* First check our per-socket dupack rate limit. */
 	if (__tcp_oow_rate_limited(net,
@@ -3627,18 +3624,22 @@ static void tcp_send_challenge_ack(struct sock *sk)
 				   &tp->last_oow_ack_time))
 		return;
 
+	ack_limit = READ_ONCE(net->ipv4.sysctl_tcp_challenge_ack_limit);
+	if (ack_limit == INT_MAX)
+		goto send_ack;
+
 	/* Then check host-wide RFC 5961 rate limit. */
 	now = jiffies / HZ;
-	if (now != READ_ONCE(challenge_timestamp)) {
-		u32 ack_limit = READ_ONCE(net->ipv4.sysctl_tcp_challenge_ack_limit);
+	if (now != READ_ONCE(net->ipv4.tcp_challenge_timestamp)) {
 		u32 half = (ack_limit + 1) >> 1;
 
-		WRITE_ONCE(challenge_timestamp, now);
-		WRITE_ONCE(challenge_count, half + prandom_u32_max(ack_limit));
+		WRITE_ONCE(net->ipv4.tcp_challenge_timestamp, now);
+		WRITE_ONCE(net->ipv4.tcp_challenge_count, half + prandom_u32_max(ack_limit));
 	}
-	count = READ_ONCE(challenge_count);
+	count = READ_ONCE(net->ipv4.tcp_challenge_count);
 	if (count > 0) {
-		WRITE_ONCE(challenge_count, count - 1);
+		WRITE_ONCE(net->ipv4.tcp_challenge_count, count - 1);
+send_ack:
 		NET_INC_STATS(net, LINUX_MIB_TCPCHALLENGEACK);
 		tcp_send_ack(sk);
 	}
net/ipv4/tcp_ipv4.c +4 -2
index 0c83780dc9bf4293135b8044daf41090b66f8b08..5b019ba2b9d2155b6d612727d32c89433d19be03 100644
@@ -3139,8 +3139,10 @@ static int __net_init tcp_sk_init(struct net *net)
 	net->ipv4.sysctl_tcp_tso_win_divisor = 3;
 	/* Default TSQ limit of 16 TSO segments */
 	net->ipv4.sysctl_tcp_limit_output_bytes = 16 * 65536;
-	/* rfc5961 challenge ack rate limiting */
-	net->ipv4.sysctl_tcp_challenge_ack_limit = 1000;
+
+	/* rfc5961 challenge ack rate limiting, per net-ns, disabled by default. */
+	net->ipv4.sysctl_tcp_challenge_ack_limit = INT_MAX;
+
 	net->ipv4.sysctl_tcp_min_tso_segs = 2;
 	net->ipv4.sysctl_tcp_tso_rtt_log = 9;  /* 2^9 = 512 usec */
 	net->ipv4.sysctl_tcp_min_rtt_wlen = 300;
Signature · 1 line
2.37.2.672.g94769d06f0-goog
maintainer Acked-by on v1 2/2
#4 lore raw

Re: [PATCH net 2/2] tcp: make global challenge ack rate limitation per net-ns and default disabled

Quoted 9 lines · Eric Dumazet

From: Eric Dumazet <[email protected]>

Because per host rate limiting has been proven problematic (side channel attacks can be based on it), per host rate limiting of challenge acks ideally should be per netns and turned off by default.

This is a long due followup of following commits:

Quoted 8 lines

083ae308280d ("tcp: enable per-socket rate limiting of all 'challenge acks'") f2b2c582e824 ("tcp: mitigate ACK loops for connections as tcp_sock") 75ff39ccc1bd ("tcp: make challenge acks less predictable")

Cc Jason Baron <[email protected]>
Cc Neal Cardwell <[email protected]>

---

Looks great. Thanks, Eric!

neal

#5 lore raw
In reply to Eric Dumazet
This series was applied to netdev/net.git (master) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net.git (master) by Jakub Kicinski <[email protected]>:

Quoted 8 lines · On Tue, 30 Aug 2022 11:56:54 -0700 you wrote:

From: Eric Dumazet <[email protected]>

syzbot found a typical data-race addressed in the first patch.

While we are at it, second patch makes the global rate limit per net-ns and disabled by default.

[...]

Here is the summary with links:
- [net,1/2] tcp: annotate data-race around challenge_timestamp

    https://git.kernel.org/netdev/net/c/8c70521238b7
  - [net,2/2] tcp: make global challenge ack rate limitation per net-ns and default disabled
    https://git.kernel.org/netdev/net/c/79e3602caa6f

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html