Skip to contentSkip to conversation

tcp: stable backports for CVE-2024-41007

PATCH In review Eric Dumazet 4 participants 11 messages Jul 16, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover tcp: stable backports for CVE-2024-41007

For linux-5.4 stable, we have to bring four patches.

(Compared to linux-5.10 where only the last two patches are needed.)

Eric Dumazet (3):
tcp: refactor tcp_retransmit_timer()
tcp: use signed arithmetic in tcp_rtx_probe0_timed_out()
tcp: avoid too many retransmit packets

Menglong Dong (1):
net: tcp: fix unexcepted socket die when snd_wnd is 0

 net/ipv4/tcp_timer.c | 45 +++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 42 insertions(+), 3 deletions(-)
Signature · 1 line
2.45.2.993.g49e7a77208-goog
#1 lore raw

PATCH 1/4 tcp: refactor tcp_retransmit_timer()

commit 0d580fbd2db084a5c96ee9c00492236a279d5e0f upstream.

It appears linux-4.14 stable needs a backport of commit 88f8598d0a30 ("tcp: exit if nothing to retransmit on RTO timeout")

Since tcp_rtx_queue_empty() is not in pre 4.15 kernels, let's refactor tcp_retransmit_timer() to only use tcp_rtx_queue_head()

I will provide to stable teams the squashed patches.

---
 net/ipv4/tcp_timer.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
net/ipv4/tcp_timer.c +8 -2
index e20fced3c9cf691c83b494e11e551161e17d6619..e0bd7999d6d51602fcd868a72d00618234bd0c27 100644
@@ -452,6 +452,7 @@ void tcp_retransmit_timer(struct sock *sk)
 	struct net *net = sock_net(sk);
 	struct inet_connection_sock *icsk = inet_csk(sk);
 	struct request_sock *req;
+	struct sk_buff *skb;
 
 	req = rcu_dereference_protected(tp->fastopen_rsk,
 					lockdep_sock_is_held(sk));
@@ -464,7 +465,12 @@ void tcp_retransmit_timer(struct sock *sk)
 		 */
 		return;
 	}
-	if (!tp->packets_out || WARN_ON_ONCE(tcp_rtx_queue_empty(sk)))
+
+	if (!tp->packets_out)
+		return;
+
+	skb = tcp_rtx_queue_head(sk);
+	if (WARN_ON_ONCE(!skb))
 		return;
 
 	if (!tp->snd_wnd && !sock_flag(sk, SOCK_DEAD) &&
@@ -496,7 +502,7 @@ void tcp_retransmit_timer(struct sock *sk)
 			goto out;
 		}
 		tcp_enter_loss(sk);
-		tcp_retransmit_skb(sk, tcp_rtx_queue_head(sk), 1);
+		tcp_retransmit_skb(sk, skb, 1);
 		__sk_dst_reset(sk);
 		goto out_reset_timer;
 	}
Signature · 1 line
2.45.2.993.g49e7a77208-goog
#2 lore raw

PATCH 2/4 net: tcp: fix unexcepted socket die when snd_wnd is 0

In reply to Eric Dumazet

From: Menglong Dong <[email protected]>

commit e89688e3e97868451a5d05b38a9d2633d6785cd4 upstream.

In tcp_retransmit_timer(), a window shrunk connection will be regarded as timeout if 'tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX'. This is not right all the time.

The retransmits will become zero-window probes in tcp_retransmit_timer() if the 'snd_wnd==0'. Therefore, the icsk->icsk_rto will come up to TCP_RTO_MAX sooner or later.

However, the timer can be delayed and be triggered after 122877ms, not TCP_RTO_MAX, as I tested.

Therefore, 'tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX' is always true once the RTO come up to TCP_RTO_MAX, and the socket will die.

Fix this by replacing the 'tcp_jiffies32' with '(u32)icsk->icsk_timeout', which is exact the timestamp of the timeout.

However, "tp->rcv_tstamp" can restart from idle, then tp->rcv_tstamp could already be a long time (minutes or hours) in the past even on the first RTO. So we double check the timeout with the duration of the retransmission.

Meanwhile, making "2 * TCP_RTO_MAX" as the timeout to avoid the socket dying too soon.

---
 net/ipv4/tcp_timer.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
net/ipv4/tcp_timer.c +17 -1
index e0bd7999d6d51602fcd868a72d00618234bd0c27..76bd619a89848a2a673b49fdb034037b454ced18 100644
@@ -434,6 +434,22 @@ static void tcp_fastopen_synack_timer(struct sock *sk, struct request_sock *req)
 			  TCP_TIMEOUT_INIT << req->num_timeout, TCP_RTO_MAX);
 }
 
+static bool tcp_rtx_probe0_timed_out(const struct sock *sk,
+				     const struct sk_buff *skb)
+{
+	const struct tcp_sock *tp = tcp_sk(sk);
+	const int timeout = TCP_RTO_MAX * 2;
+	u32 rcv_delta, rtx_delta;
+
+	rcv_delta = inet_csk(sk)->icsk_timeout - tp->rcv_tstamp;
+	if (rcv_delta <= timeout)
+		return false;
+
+	rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
+			(tp->retrans_stamp ?: tcp_skb_timestamp(skb)));
+
+	return rtx_delta > timeout;
+}
 
 /**
  *  tcp_retransmit_timer() - The TCP retransmit timeout handler
@@ -497,7 +513,7 @@ void tcp_retransmit_timer(struct sock *sk)
 					    tp->snd_una, tp->snd_nxt);
 		}
 #endif
-		if (tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX) {
+		if (tcp_rtx_probe0_timed_out(sk, skb)) {
 			tcp_write_err(sk);
 			goto out;
 		}
Signature · 1 line
2.45.2.993.g49e7a77208-goog
#3 lore raw

PATCH 3/4 tcp: use signed arithmetic in tcp_rtx_probe0_timed_out()

In reply to Eric Dumazet

commit 36534d3c54537bf098224a32dc31397793d4594d upstream.

Due to timer wheel implementation, a timer will usually fire after its schedule.

For instance, for HZ=1000, a timeout between 512ms and 4s has a granularity of 64ms.
For this range of values, the extra delay could be up to 63ms.

For TCP, this means that tp->rcv_tstamp may be after inet_csk(sk)->icsk_timeout whenever the timer interrupt finally triggers, if one packet came during the extra delay.

We need to make sure tcp_rtx_probe0_timed_out() handles this case.

Fixes e89688e3e978 ("net: tcp: fix unexcepted socket die when snd_wnd is 0")
Cc Menglong Dong <[email protected]>
---
 net/ipv4/tcp_timer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
net/ipv4/tcp_timer.c +6 -1
index 76bd619a89848a2a673b49fdb034037b454ced18..cbd4fde47c1f8d29533bf5ce28bddf4c9a00efe7 100644
@@ -439,8 +439,13 @@ static bool tcp_rtx_probe0_timed_out(const struct sock *sk,
 {
 	const struct tcp_sock *tp = tcp_sk(sk);
 	const int timeout = TCP_RTO_MAX * 2;
-	u32 rcv_delta, rtx_delta;
+	u32 rtx_delta;
+	s32 rcv_delta;
 
+	/* Note: timer interrupt might have been delayed by at least one jiffy,
+	 * and tp->rcv_tstamp might very well have been written recently.
+	 * rcv_delta can thus be negative.
+	 */
 	rcv_delta = inet_csk(sk)->icsk_timeout - tp->rcv_tstamp;
 	if (rcv_delta <= timeout)
 		return false;
Signature · 1 line
2.45.2.993.g49e7a77208-goog
#4 lore raw

PATCH 4/4 tcp: avoid too many retransmit packets

In reply to Eric Dumazet

commit 97a9063518f198ec0adb2ecb89789de342bb8283 upstream.

If a TCP socket is using TCP_USER_TIMEOUT, and the other peer retracted its window to zero, tcp_retransmit_timer() can retransmit a packet every two jiffies (2 ms for HZ=1000), for about 4 minutes after TCP_USER_TIMEOUT has 'expired'.

The fix is to make sure tcp_rtx_probe0_timed_out() takes icsk->icsk_user_timeout into account.

Before blamed commit, the socket would not timeout after icsk->icsk_user_timeout, but would use standard exponential backoff for the retransmits.

Also worth noting that before commit e89688e3e978 ("net: tcp:
fix unexcepted socket die when snd_wnd is 0"), the issue would last 2 minutes instead of 4.

Fixes b701a99e431d ("tcp: Add tcp_clamp_rto_to_user_timeout() helper to improve accuracy")
Cc Neal Cardwell <[email protected]>
---
 net/ipv4/tcp_timer.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)
net/ipv4/tcp_timer.c +14 -2
index cbd4fde47c1f8d29533bf5ce28bddf4c9a00efe7..a386e9b84984ab0be41b0c38ea015e4ae3377edf 100644
@@ -437,16 +437,28 @@ static void tcp_fastopen_synack_timer(struct sock *sk, struct request_sock *req)
 static bool tcp_rtx_probe0_timed_out(const struct sock *sk,
 				     const struct sk_buff *skb)
 {
+	const struct inet_connection_sock *icsk = inet_csk(sk);
+	u32 user_timeout = READ_ONCE(icsk->icsk_user_timeout);
 	const struct tcp_sock *tp = tcp_sk(sk);
-	const int timeout = TCP_RTO_MAX * 2;
+	int timeout = TCP_RTO_MAX * 2;
 	u32 rtx_delta;
 	s32 rcv_delta;
 
+	if (user_timeout) {
+		/* If user application specified a TCP_USER_TIMEOUT,
+		 * it does not want win 0 packets to 'reset the timer'
+		 * while retransmits are not making progress.
+		 */
+		if (rtx_delta > user_timeout)
+			return true;
+		timeout = min_t(u32, timeout, msecs_to_jiffies(user_timeout));
+	}
+
 	/* Note: timer interrupt might have been delayed by at least one jiffy,
 	 * and tp->rcv_tstamp might very well have been written recently.
 	 * rcv_delta can thus be negative.
 	 */
-	rcv_delta = inet_csk(sk)->icsk_timeout - tp->rcv_tstamp;
+	rcv_delta = icsk->icsk_timeout - tp->rcv_tstamp;
 	if (rcv_delta <= timeout)
 		return false;
 
Signature · 1 line
2.45.2.993.g49e7a77208-goog
#5 lore raw

Re: [PATCH stable-5.4 4/4] tcp: avoid too many retransmit packets

Hi Greg, Eric, all,

I noticed this in stable-rc/queue and stable-rc/linux- for 6.1 and 6.6:

    net/ipv4/tcp_timer.c:472:7: error: variable 'rtx_delta' is uninitialized when used here [-Werror,-Wuninitialized]
                    if (rtx_delta > user_timeout)
                        ^~~~~~~~~
    net/ipv4/tcp_timer.c:464:15: note: initialize the variable 'rtx_delta' to silence this warning
            u32 rtx_delta;
                        ^
                        = 0

I hope that helps!

Cheers,
Miguel

#6 lore raw

Re: [PATCH stable-5.4 4/4] tcp: avoid too many retransmit packets

Quoted 13 lines · Miguel Ojeda

Hi Greg, Eric, all,

I noticed this in stable-rc/queue and stable-rc/linux- for 6.1 and 6.6:

    net/ipv4/tcp_timer.c:472:7: error: variable 'rtx_delta' is uninitialized when used here [-Werror,-Wuninitialized]
                    if (rtx_delta > user_timeout)
                        ^~~~~~~~~
    net/ipv4/tcp_timer.c:464:15: note: initialize the variable 'rtx_delta' to silence this warning
            u32 rtx_delta;
                        ^
                        = 0
Quoted 1 line

I hope that helps!

Thanks for the report!

I think it missed one small snippet of code from [1] compared to the latest kernel. We can init this part before using it, something like this:

+ rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
+ (tp->retrans_stamp ?: tcp_skb_timestamp(skb)));

Note: fully untested.

Since Eric is very busy, I decided to check and provide some useful information here.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=614e8316aa4cafba3e204cb8ee48bd12b92f3d93

Quoted 3 lines

Cheers,
Miguel

#7 lore raw

Re: [PATCH stable-5.4 4/4] tcp: avoid too many retransmit packets

Quoted 25 lines · Jason Xing
Quoted 13 lines · Miguel Ojeda

Hi Greg, Eric, all,

I noticed this in stable-rc/queue and stable-rc/linux- for 6.1 and 6.6:

    net/ipv4/tcp_timer.c:472:7: error: variable 'rtx_delta' is uninitialized when used here [-Werror,-Wuninitialized]
                    if (rtx_delta > user_timeout)
                        ^~~~~~~~~
    net/ipv4/tcp_timer.c:464:15: note: initialize the variable 'rtx_delta' to silence this warning
            u32 rtx_delta;
                        ^
                        = 0
Quoted 1 line

I hope that helps!

Thanks for the report!

I think it missed one small snippet of code from [1] compared to the latest kernel. We can init this part before using it, something like this:

(quoted diff) +2 -0
+       rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
+                       (tp->retrans_stamp ?: tcp_skb_timestamp(skb)));
Quoted 4 lines

Note: fully untested.

Since Eric is very busy, I decided to check and provide some useful information here.

Thanks all, this was probably due to my manual backporting here, let me go check what went wrong...

#8 lore raw

Re: [PATCH stable-5.4 4/4] tcp: avoid too many retransmit packets

Quoted 26 lines · Greg KH
Quoted 25 lines · Jason Xing
Quoted 13 lines · Miguel Ojeda

Hi Greg, Eric, all,

I noticed this in stable-rc/queue and stable-rc/linux- for 6.1 and 6.6:

    net/ipv4/tcp_timer.c:472:7: error: variable 'rtx_delta' is uninitialized when used here [-Werror,-Wuninitialized]
                    if (rtx_delta > user_timeout)
                        ^~~~~~~~~
    net/ipv4/tcp_timer.c:464:15: note: initialize the variable 'rtx_delta' to silence this warning
            u32 rtx_delta;
                        ^
                        = 0
Quoted 1 line

I hope that helps!

Thanks for the report!

I think it missed one small snippet of code from [1] compared to the latest kernel. We can init this part before using it, something like this:

(quoted diff) +2 -0
+       rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
+                       (tp->retrans_stamp ?: tcp_skb_timestamp(skb)));
Quoted 7 lines
Quoted 4 lines

Note: fully untested.

Since Eric is very busy, I decided to check and provide some useful information here.

Thanks all, this was probably due to my manual backporting here, let me go check what went wrong...

Yeah, this is my fault, due to 614e8316aa4c ("tcp: add support for usec resolution in TCP TS values") not being in the tree, let me go rework things...

thanks,

greg k-h

#9 lore raw

Re: [PATCH stable-5.4 4/4] tcp: avoid too many retransmit packets

Quoted 32 lines · Greg KH
Quoted 31 lines · Greg KH
Quoted 29 lines · Jason Xing
Quoted 13 lines · Miguel Ojeda

Hi Greg, Eric, all,

I noticed this in stable-rc/queue and stable-rc/linux- for 6.1 and 6.6:

    net/ipv4/tcp_timer.c:472:7: error: variable 'rtx_delta' is uninitialized when used here [-Werror,-Wuninitialized]
                    if (rtx_delta > user_timeout)
                        ^~~~~~~~~
    net/ipv4/tcp_timer.c:464:15: note: initialize the variable 'rtx_delta' to silence this warning
            u32 rtx_delta;
                        ^
                        = 0
Quoted 1 line

I hope that helps!

Thanks for the report!

I think it missed one small snippet of code from [1] compared to the latest kernel. We can init this part before using it, something like this:

(quoted diff) +2 -0
+       rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
+                       (tp->retrans_stamp ?: tcp_skb_timestamp(skb)));

Note: fully untested.

Since Eric is very busy, I decided to check and provide some useful information here.

Quoted 6 lines
Quoted 2 lines

Thanks all, this was probably due to my manual backporting here, let me go check what went wrong...

Yeah, this is my fault, due to 614e8316aa4c ("tcp: add support for usec resolution in TCP TS values") not being in the tree, let me go rework things...

Ok, backporting that commit is not going to happen, that's crazy...

Anyway, the diff below is what I made on top of the existing one, which should be doing the right thing. But ideally someone can test this, somehow... I'll push out -rc releases later today so that people can pound on it easier.

thanks for the review!

greg k-h

net/ipv4/tcp_timer.c +3 -3
@@ -464,6 +464,9 @@ static bool tcp_rtx_probe0_timed_out(con
 	u32 rtx_delta;
 	s32 rcv_delta;
 
+	rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
+			(tp->retrans_stamp ?: tcp_skb_timestamp(skb)));
+
 	if (user_timeout) {
 		/* If user application specified a TCP_USER_TIMEOUT,
 		 * it does not want win 0 packets to 'reset the timer'
@@ -482,9 +485,6 @@ static bool tcp_rtx_probe0_timed_out(con
 	if (rcv_delta <= timeout)
 		return false;
 
-	rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
-			(tp->retrans_stamp ?: tcp_skb_timestamp(skb)));
-
 	return rtx_delta > timeout;
 }
 
#10 lore raw

Re: [PATCH stable-5.4 4/4] tcp: avoid too many retransmit packets

Quoted 34 lines · Greg KH
Quoted 32 lines · Greg KH
Quoted 31 lines · Greg KH
Quoted 29 lines · Jason Xing
Quoted 13 lines · Miguel Ojeda

Hi Greg, Eric, all,

I noticed this in stable-rc/queue and stable-rc/linux- for 6.1 and 6.6:

    net/ipv4/tcp_timer.c:472:7: error: variable 'rtx_delta' is uninitialized when used here [-Werror,-Wuninitialized]
                    if (rtx_delta > user_timeout)
                        ^~~~~~~~~
    net/ipv4/tcp_timer.c:464:15: note: initialize the variable 'rtx_delta' to silence this warning
            u32 rtx_delta;
                        ^
                        = 0
Quoted 1 line

I hope that helps!

Thanks for the report!

I think it missed one small snippet of code from [1] compared to the latest kernel. We can init this part before using it, something like this:

net/ipv4/tcp_timer.c +2 -0
+       rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
+                       (tp->retrans_stamp ?: tcp_skb_timestamp(skb)));

Note: fully untested.

Since Eric is very busy, I decided to check and provide some useful information here.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Thanks all, this was probably due to my manual backporting here, let me go check what went wrong...

Yeah, this is my fault, due to 614e8316aa4c ("tcp: add support for usec resolution in TCP TS values") not being in the tree, let me go rework things...

Ok, backporting that commit is not going to happen, that's crazy...

Absolutely right, this is not stable material.

Quoted 24 lines

Anyway, the diff below is what I made on top of the existing one, which should be doing the right thing. But ideally someone can test this, somehow... I'll push out -rc releases later today so that people can pound on it easier.

thanks for the review!

greg k-h

net/ipv4/tcp_timer.c +3 -0
@@ -464,6 +464,9 @@ static bool tcp_rtx_probe0_timed_out(con
        u32 rtx_delta;
        s32 rcv_delta;
 
+       rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
+                       (tp->retrans_stamp ?: tcp_skb_timestamp(skb)));
+
        if (user_timeout) {
                /* If user application specified a TCP_USER_TIMEOUT,
                 * it does not want win 0 packets to 'reset the timer'

net/ipv4/tcp_timer.c static bool tcp_rtx_probe0_timed_out(con · line 490

Quoted 9 lines
net/ipv4/tcp_timer.c +0 -3
@@ -482,9 +485,6 @@ static bool tcp_rtx_probe0_timed_out(con
        if (rcv_delta <= timeout)
                return false;
 
-       rtx_delta = (u32)msecs_to_jiffies(tcp_time_stamp(tp) -
-                       (tp->retrans_stamp ?: tcp_skb_timestamp(skb)));
-
        return rtx_delta > timeout;
 }
 

Hi Greg, this looks great to me, thanks for taking care of this.