Skip to contentSkip to conversation

tcp: remove bad timeout logic in fast recovery

PATCH Applied Yuchung Cheng 5 participants 5 messages May 17, 2013 – May 20, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tcp: remove bad timeout logic in fast recovery

tcp_timeout_skb() was intended to trigger fast recovery on timeout, unfortunately in reality it often causes spurious retransmission storms during fast recovery. The particular sign is fast retransmit over highest sacked sequence (SND.FACK).

Currently the RTO timer re-arming (as in RFC6298) offers a nice cushion to avoid spurious timeout: when SND.UNA advances the sender re-arms RTO and extends the timeout by icsk_rto. The sender does not offset the time elapsed since the packet at SND.UNA was sent.

But if the next (DUP)ACK arrives later than ~RTTVAR and triggers tcp_fastretrans_alert(), then tcp_timeout_skb() will mark any packet sent before icsk_rto interval lost, including one that's above the highest sacked sequence. Most likely a large part of scorebard will be marked.

If most packets are not lost then the subsequence DUPACKs with new SACK blockes will cause the sender to continue retransmit packets beyond SND.FACK spuriously right. Even only one packet is lost the sender may falsely retransmit almost the entire window.

The situation becomes common in the world of bufferbloat: the RTT continues to grow as the queue builds up but RTTVAR remains small and close to the minimum 200ms. If a data packet is lost and the DUPACK triggered by the next data packet is slightly delayed, then a spurious retransmission storm forms.

As the original comment on tcp_timeout_skb() suggests: the usefulness of this feature is questionable. It also wastes cycles walking the sack scoreboard and is actually harmful because of the false recovery. It's time to remove this.

---
 include/linux/tcp.h  |  1 -
 include/net/tcp.h    |  1 -
 net/ipv4/tcp_input.c | 65 +---------------------------------------------------
 3 files changed, 1 insertion(+), 66 deletions(-)
include/linux/tcp.h +0 -1
index 5adbc33..472120b 100644
@@ -246,7 +246,6 @@ struct tcp_sock {
 
 	/* from STCP, retrans queue hinting */
 	struct sk_buff* lost_skb_hint;
-	struct sk_buff *scoreboard_skb_hint;
 	struct sk_buff *retransmit_skb_hint;
 
 	struct sk_buff_head	out_of_order_queue; /* Out of order segments go here */
include/net/tcp.h +0 -1
index 5bba80f..e1c3723 100644
@@ -1193,7 +1193,6 @@ static inline void tcp_mib_init(struct net *net)
 static inline void tcp_clear_retrans_hints_partial(struct tcp_sock *tp)
 {
 	tp->lost_skb_hint = NULL;
-	tp->scoreboard_skb_hint = NULL;
 }
 
 static inline void tcp_clear_all_retrans_hints(struct tcp_sock *tp)
net/ipv4/tcp_input.c +1 -64

107 lines not on this page · raw message

Signature · 1 line
1.8.2.1
Acked-by on v1 1/1
#1 lore raw
Quoted 32 lines · Yuchung Cheng

tcp_timeout_skb() was intended to trigger fast recovery on timeout, unfortunately in reality it often causes spurious retransmission storms during fast recovery. The particular sign is fast retransmit over highest sacked sequence (SND.FACK).

Currently the RTO timer re-arming (as in RFC6298) offers a nice cushion to avoid spurious timeout: when SND.UNA advances the sender re-arms RTO and extends the timeout by icsk_rto. The sender does not offset the time elapsed since the packet at SND.UNA was sent.

But if the next (DUP)ACK arrives later than ~RTTVAR and triggers tcp_fastretrans_alert(), then tcp_timeout_skb() will mark any packet sent before icsk_rto interval lost, including one that's above the highest sacked sequence. Most likely a large part of scorebard will be marked.

If most packets are not lost then the subsequence DUPACKs with new SACK blockes will cause the sender to continue retransmit packets beyond SND.FACK spuriously right. Even only one packet is lost the sender may falsely retransmit almost the entire window.

The situation becomes common in the world of bufferbloat: the RTT continues to grow as the queue builds up but RTTVAR remains small and close to the minimum 200ms. If a data packet is lost and the DUPACK triggered by the next data packet is slightly delayed, then a spurious retransmission storm forms.

As the original comment on tcp_timeout_skb() suggests: the usefulness of this feature is questionable. It also wastes cycles walking the sack scoreboard and is actually harmful because of the false recovery. It's time to remove this.

Quoted 6 lines
---
 include/linux/tcp.h  |  1 -
 include/net/tcp.h    |  1 -
 net/ipv4/tcp_input.c | 65 +---------------------------------------------------
 3 files changed, 1 insertion(+), 66 deletions(-)
maintainer Acked-by
#2 lore raw
Quoted 33 lines · Eric Dumazet
Quoted 32 lines · Yuchung Cheng

tcp_timeout_skb() was intended to trigger fast recovery on timeout, unfortunately in reality it often causes spurious retransmission storms during fast recovery. The particular sign is fast retransmit over highest sacked sequence (SND.FACK).

Currently the RTO timer re-arming (as in RFC6298) offers a nice cushion to avoid spurious timeout: when SND.UNA advances the sender re-arms RTO and extends the timeout by icsk_rto. The sender does not offset the time elapsed since the packet at SND.UNA was sent.

But if the next (DUP)ACK arrives later than ~RTTVAR and triggers tcp_fastretrans_alert(), then tcp_timeout_skb() will mark any packet sent before icsk_rto interval lost, including one that's above the highest sacked sequence. Most likely a large part of scorebard will be marked.

If most packets are not lost then the subsequence DUPACKs with new SACK blockes will cause the sender to continue retransmit packets beyond SND.FACK spuriously right. Even only one packet is lost the sender may falsely retransmit almost the entire window.

The situation becomes common in the world of bufferbloat: the RTT continues to grow as the queue builds up but RTTVAR remains small and close to the minimum 200ms. If a data packet is lost and the DUPACK triggered by the next data packet is slightly delayed, then a spurious retransmission storm forms.

As the original comment on tcp_timeout_skb() suggests: the usefulness of this feature is questionable. It also wastes cycles walking the sack scoreboard and is actually harmful because of the false recovery. It's time to remove this.

Quoted 8 lines
Quoted 6 lines
---
 include/linux/tcp.h  |  1 -
 include/net/tcp.h    |  1 -
 net/ipv4/tcp_input.c | 65 +---------------------------------------------------
 3 files changed, 1 insertion(+), 66 deletions(-)

neal

#3 lore raw
Quoted 41 lines · Neal Cardwell
Quoted 40 lines · Eric Dumazet
Quoted 38 lines · Yuchung Cheng

tcp_timeout_skb() was intended to trigger fast recovery on timeout, unfortunately in reality it often causes spurious retransmission storms during fast recovery. The particular sign is fast retransmit over highest sacked sequence (SND.FACK).

Currently the RTO timer re-arming (as in RFC6298) offers a nice cushion to avoid spurious timeout: when SND.UNA advances the sender re-arms RTO and extends the timeout by icsk_rto. The sender does not offset the time elapsed since the packet at SND.UNA was sent.

But if the next (DUP)ACK arrives later than ~RTTVAR and triggers tcp_fastretrans_alert(), then tcp_timeout_skb() will mark any packet sent before icsk_rto interval lost, including one that's above the highest sacked sequence. Most likely a large part of scorebard will be marked.

If most packets are not lost then the subsequence DUPACKs with new SACK blockes will cause the sender to continue retransmit packets beyond SND.FACK spuriously right. Even only one packet is lost the sender may falsely retransmit almost the entire window.

The situation becomes common in the world of bufferbloat: the RTT continues to grow as the queue builds up but RTTVAR remains small and close to the minimum 200ms. If a data packet is lost and the DUPACK triggered by the next data packet is slightly delayed, then a spurious retransmission storm forms.

As the original comment on tcp_timeout_skb() suggests: the usefulness of this feature is questionable. It also wastes cycles walking the sack scoreboard and is actually harmful because of the false recovery. It's time to remove this.

---
 include/linux/tcp.h  |  1 -
 include/net/tcp.h    |  1 -
 net/ipv4/tcp_input.c | 65 +---------------------------------------------------
 3 files changed, 1 insertion(+), 66 deletions(-)
Quoted 3 lines
Quoted 1 line
maintainer Applied
#4 lore raw

From: Nandita Dukkipati <[email protected]>
Date: Fri, 17 May 2013 22:52:34 -0700

Quoted 42 lines · Neal Cardwell
Quoted 41 lines · Eric Dumazet
Quoted 40 lines · Yuchung Cheng
Quoted 38 lines

tcp_timeout_skb() was intended to trigger fast recovery on timeout, unfortunately in reality it often causes spurious retransmission storms during fast recovery. The particular sign is fast retransmit over highest sacked sequence (SND.FACK).

Currently the RTO timer re-arming (as in RFC6298) offers a nice cushion to avoid spurious timeout: when SND.UNA advances the sender re-arms RTO and extends the timeout by icsk_rto. The sender does not offset the time elapsed since the packet at SND.UNA was sent.

But if the next (DUP)ACK arrives later than ~RTTVAR and triggers tcp_fastretrans_alert(), then tcp_timeout_skb() will mark any packet sent before icsk_rto interval lost, including one that's above the highest sacked sequence. Most likely a large part of scorebard will be marked.

If most packets are not lost then the subsequence DUPACKs with new SACK blockes will cause the sender to continue retransmit packets beyond SND.FACK spuriously right. Even only one packet is lost the sender may falsely retransmit almost the entire window.

The situation becomes common in the world of bufferbloat: the RTT continues to grow as the queue builds up but RTTVAR remains small and close to the minimum 200ms. If a data packet is lost and the DUPACK triggered by the next data packet is slightly delayed, then a spurious retransmission storm forms.

As the original comment on tcp_timeout_skb() suggests: the usefulness of this feature is questionable. It also wastes cycles walking the sack scoreboard and is actually harmful because of the false recovery. It's time to remove this.

---
 include/linux/tcp.h  |  1 -
 include/net/tcp.h    |  1 -
 net/ipv4/tcp_input.c | 65 +---------------------------------------------------
 3 files changed, 1 insertion(+), 66 deletions(-)
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Applied, thanks everyone.