apply cwnd rules to FIN packets with data
Subsystems (3)
PATCH 1/1 apply cwnd rules to FIN packets with data
This is especially important with TSO enabled. Currently, it will send a burst of up to 64k at the end of a connection, even when cwnd is much smaller than 64k. This patch still lets out empty FIN packets, but does not apply the special case to FINs carrying data.
-John
Apply cwnd rules to FIN packets that contain data.
---
commit af319609eee705e0791a1a58c33b216e8d0254bf
tree 5a1afcc506e09f5adfd74efb7e0cbbc82ec4d5b0
parent c0d4d573feed199b16094c072e7cb07afb01c598
author John Heffner <[email protected]> Mon, 05 Feb 2007 16:25:46 -0500 committer John Heffner <[email protected]> Mon, 05 Feb 2007 16:25:46 -0500
net/ipv4/tcp_output.c | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-)
net/ipv4/tcp_output.c +2 -5
@@ -965,7 +965,7 @@ static inline unsigned int tcp_cwnd_testu32 in_flight, cwnd;/* Don't be strict about the congestion window for the final FIN. */ -if (TCP_SKB_CB (skb)->flags &TCPCB_FLAG_FIN ) +if ((TCP_SKB_CB (skb)->flags &TCPCB_FLAG_FIN ) && skb->len ==0 )return 1 ; in_flight =tcp_packets_in_flight (tp);
@@ -1034,7 +1034,7 @@ static inline int tcp_nagle_test(struct/* Don't use the nagle rule for urgent data (or for the final FIN). */ if (tp->urg_mode || - (TCP_SKB_CB (skb)->flags &TCPCB_FLAG_FIN )) + ((TCP_SKB_CB (skb)->flags &TCPCB_FLAG_FIN ) && skb->len ==0 ))return 1 ;if (!tcp_nagle_check (tp, skb, cur_mss, nonagle))
@@ -1156,9 +1156,6 @@ static int tcp_tso_should_defer(struct sconst struct inet_connection_sock *icsk =inet_csk (sk);u32 send_win, cong_win, limit, in_flight; -if (TCP_SKB_CB (skb)->flags &TCPCB_FLAG_FIN ) - goto send_now; -if (icsk->icsk_ca_state != TCP_CA_Open) goto send_now;
David Miller
maintainerFrom: John Heffner <[email protected]>
Date: Mon, 05 Feb 2007 16:58:18 -0500
Quoted 4 lines
This is especially important with TSO enabled. Currently, it will send a burst of up to 64k at the end of a connection, even when cwnd is much smaller than 64k. This patch still lets out empty FIN packets, but does not apply the special case to FINs carrying data.
Good catch John.
But I think the correct test on skb->len would be to just make sure that it is <= REAL_MSS.
What do you think about that? This would match the original intention of the logic in the pre-TSO days.
David Miller
maintainerFrom: John Heffner <[email protected]>
Date: Mon, 05 Feb 2007 18:02:19 -0500
Quoted 11 lines
Quoted 10 lines · David Miller
From: John Heffner <[email protected]>
Date: Mon, 05 Feb 2007 16:58:18 -0500Quoted 4 lines
This is especially important with TSO enabled. Currently, it will send a burst of up to 64k at the end of a connection, even when cwnd is much smaller than 64k. This patch still lets out empty FIN packets, but does not apply the special case to FINs carrying data.
Good catch John.
Quoted 7 lines
Quoted 5 lines
But I think the correct test on
skb->lenwould be to just make sure that it is <=REAL_MSS.What do you think about that? This would match the original intention of the logic in the pre-TSO days.
What was the intention of that logic?
Because a packet is a packet is a packet.
If we let a FIN out it's basically equivalent to a FIN+data as far as the routers are concerned. Either they will drop the packet or they won't. And they will do this regardless of whether data is attached to the FIN.
Getting FINs out fast is important for another reason, the faster the session closes the faster "smart" intermediate routers can teardown cached routing or firewall tracking entries for the flow.
Quoted 4 lines
Actually, I think it would be better to leave the Nagle test as it was (which is implicitly <
real_mss), because there is obviously no point in doing the nagle test when you know there is no more data that will be sent.
True.
Quoted 2 lines
However, I can't think of any reason why the cwnd test should not apply.
Care to elaborate here? You can view the FIN special case as an off by one error in the CWND test, it's not going to melt the internet. :-)
Quoted 5 lines · David Miller
From: John Heffner <[email protected]>
Date: Mon, 05 Feb 2007 16:58:18 -0500Quoted 1 line
This is especially important with TSO enabled. Currently, it will send a burst of up to 64k at the end of a connection, even when cwnd is much smaller than 64k. This patch still lets out empty FIN packets, but does not apply the special case to FINs carrying data.
Quoted 7 lines
Good catch John.
But I think the correct test on
skb->lenwould be to just make sure that it is <=REAL_MSS.What do you think about that? This would match the original intention of the logic in the pre-TSO days.
What was the intention of that logic?
Actually, I think it would be better to leave the Nagle test as it was (which is implicitly < real_mss), because there is obviously no point in doing the nagle test when you know there is no more data that will be sent. However, I can't think of any reason why the cwnd test should not apply.
-John
Quoted 5 lines · David Miller
Quoted 1 line
However, I can't think of any reason why the cwnd test should not apply.
Care to elaborate here? You can view the FIN special case as an off by one error in the CWND test, it's not going to melt the internet. :-)
True, it's not going to melt the internet, but why stop at one when two would finish the connection even faster? Not sure I buy this argument.
Was there some benchmarking data that was a justification for this in
the first place?
My first patch was broken anyway (should not have pulled the test from tso_should_defer), and the change is not needed to the nagle test since it's implicit. This patch just restores the old behavior from before TSO, sending the FIN when it's the last true segment. We can debate the merits of applying congestion control to the FIN separately. :)
-John
Don't apply FIN exception to full TSO segments.
---
commit 89de0d8cb75958b0315c076b31a597143e30f7a4
tree 7e9c321e62729c6ef76e3886fe9edf2ac78a680c
parent c0d4d573feed199b16094c072e7cb07afb01c598
author John Heffner <[email protected]> Mon, 05 Feb 2007 18:42:31 -0500 committer John Heffner <[email protected]> Mon, 05 Feb 2007 18:42:31 -0500
net/ipv4/tcp_output.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
net/ipv4/tcp_output.c +2 -1
@@ -965,7 +965,8 @@ static inline unsigned int tcp_cwnd_testu32 in_flight, cwnd;/* Don't be strict about the congestion window for the final FIN. */ -if (TCP_SKB_CB (skb)->flags &TCPCB_FLAG_FIN ) +if ((TCP_SKB_CB (skb)->flags &TCPCB_FLAG_FIN ) && +tcp_skb_pcount (skb) ==1 )return 1 ; in_flight =tcp_packets_in_flight (tp);
Quoted 5 lines · John Heffner
Quoted 3 lines · David Miller
Quoted 1 line
However, I can't think of any reason why the cwnd test should not apply.
Quoted 8 lines
Quoted 3 lines
Care to elaborate here? You can view the FIN special case as an off by one error in the CWND test, it's not going to melt the internet. :-)
True, it's not going to melt the internet, but why stop at one when two would finish the connection even faster? Not sure I buy this argument.
Was there some benchmarking data that was a justification for this in
the first place?
Is the cwnd in the stack byte based, or packet based?
While "all" the RFCs tend to discuss things in terms of byte-based cwnds and assumptions based on MSSes and such, the underlying principle was/is a conservation of packets. As David said, a packet is a packet, and if one were going to be sending a FIN segment, it might as well carry data.
And if one isn't comfortable sending that one last data segment with
the FIN because cwnd wasn't large enough at the time, should the FIN be sent at that point, even if it is waffer thin?
rick jones
2 cents tossed-in from the peanut gallery
Quoted 11 lines · Rick Jones
Quoted 10 lines · John Heffner
Quoted 6 lines · David Miller
Quoted 1 line
However, I can't think of any reason why the cwnd test should not apply.
Care to elaborate here? You can view the FIN special case as an off by one error in the CWND test, it's not going to melt the internet. :-)
Quoted 7 lines
Quoted 1 line
True, it's not going to melt the internet, but why stop at one when two would finish the connection even faster? Not sure I buy this argument. Was there some benchmarking data that was a justification for this in the first place?
Is the cwnd in the stack byte based, or packet based?
While "all" the RFCs tend to discuss things in terms of byte-based cwnds and assumptions based on MSSes and such, the underlying principle was/is a conservation of packets. As David said, a packet is a packet, and if one were going to be sending a FIN segment, it might as well carry data.
And if one isn't comfortable sending that one last data segment with
the FIN because cwnd wasn't large enough at the time, should the FIN be sent at that point, even if it is waffer thin?
The most conservative thing is to apply congestion control exactly as you would to any other segment, that is, just take the special case out entirely. An empty FIN is not too likely to cause problems, a full-MSS FIN somewhat more so, 2-MSS, yet more, a 64k TSO segment even more. :) I don't have hard data to argue for or against any particular optimization, but it seems there should be some if we're ignoring the standard cwnd rules.
-John
David Miller
maintainerFrom: John Heffner <[email protected]>
Date: Mon, 05 Feb 2007 19:11:09 -0500
Quoted 5 lines
My first patch was broken anyway (should not have pulled the test from
tso_should_defer), and the change is not needed to the nagle test since it's implicit. This patch just restores the old behavior from before TSO, sending the FIN when it's the last true segment. We can debate the merits of applying congestion control to the FIN separately. :)
Yes, let's split one hair at a time :-)
Patch applied, thanks John. I'll push this to the 2.6.x -stable branch too.