TCP congestion graphs
Subsystems (3)
I ran some congestion window tests against 2.6.19-rc3. For congestion window graphs see:
http://developer.osdl.org/shemminger/tcp/2.6.19-rc3/
The connection was a single flow with a 500ms RTT and a 100Mbit slowest link speed.
BIC OK
CUBIC OK (after patch)
HIGHSPEED BROKEN?
HTCP OK
RENO OK (massive overshoot)
SCALEABLE OK
VEGAS OK (no better than Reno)
VENO Same as Reno?
WESTWOOD Overshoot then low window
Highspeed has something wrong (test glitch?) Westwood behaves poorly
Veno seems no better than either Reno or Vegas in this case.
The end systems were from latest git. The netem bridge is using 2.6.18-rt with a patch to netem to use hrtimers. iperf was modified to allow easy selection of congestion control.
Signature · 1 line
Stephen Hemminger <[email protected]>
Not sure why the slow start for cubic is slower than the others. We will check on this.
----- Original Message ----- From: "Stephen Hemminger" <[email protected]>
To: "Douglas Leith" <[email protected]>; "Sangtae Ha" <[email protected]>
Cc: <[email protected]>
Sent: Wednesday, October 25, 2006 2:02 PM
Subject: TCP congestion graphs
Quoted 32 lines
I ran some congestion window tests against 2.6.19-rc3. For congestion window graphs see:
http://developer.osdl.org/shemminger/tcp/2.6.19-rc3/The connection was a single flow with a 500ms RTT and a 100Mbit slowest link speed.
BIC OK
CUBIC OK (after patch)
HIGHSPEED BROKEN?
HTCP OK
RENO OK (massive overshoot)
SCALEABLE OK
VEGAS OK (no better than Reno)
VENO Same as Reno?
WESTWOOD Overshoot then low windowHighspeed has something wrong (test glitch?) Westwood behaves poorly
Veno seems no better than either Reno or Vegas in this case.The end systems were from latest git. The netem bridge is using 2.6.18-rt with a patch to netem to use hrtimers. iperf was modified to allow easy selection of congestion control.
Signature · 5 lines
Stephen Hemminger <[email protected]> - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 3 lines · On Wed, 25 Oct 2006 18:34:17 -0400 "Injong Rhee" <[email protected]> wrote:
Not sure why the slow start for cubic is slower than the others. We will check on this.
I think it is because cubic initializes with a ssthresh of 100, and others leave ssthresh uninitialized until the first loss. This causes cubic to change to conservative mode once it crosses the initial ssthresh. Other protocols just keep doing additive increase.
My large BDP test has a 500ms delay (250ms each way) and a real 100mbit link on the last hop.
Because the intermediate emulator has a large queue (10,000 packets) large growth is possible before the first loss event. There were also no other flows during this test to fill the intermediate queue.
This may very well be a bogus test, I just was looking for large BDP behavior quirks. Any test without background traffic as you have argued is unrealistic.
Hi Stephen,
is your rt-patch to netem public available?
Best regards
HGN
Signature · 3 lines
Signed and/or encrypted mails preferd. Key-Id = 0x98350C22 Fingerprint = 490F 557B 6C48 6D7E 5706 2EA2 4A22 8D45 9835 0C22 Key available under: www.jauu.net/download/gnupg_key
Quoted 8 lines · On Thu, 26 Oct 2006 20:50:19 +0200 Hagen Paul Pfeifer <[email protected]> wrote:
Hi Stephen,
is your rt-patch to netem public available?
Best regards
HGN
The tools are in the tcp directory
http://developer.osdl.org/shemminger/tcp/netem-2.6.18-rt.patch
Signature · 1 line
Stephen Hemminger <[email protected]>
TCP congestion graphs (2.6.19-rc4)
In reply to Injong RheeI reran the tests, this time using my DSL line to home (80ms 1Mbit/sec).
http://developer.osdl.org/shemminger/tcp/2.6.19-rc4/dsl
The only obvious problem is Vegas looks broken.
Re: TCP congestion graphs (2.6.19-rc4)
Quoted 4 lines · Stephen Hemminger
I reran the tests, this time using my DSL line to home (80ms 1Mbit/sec).
http://developer.osdl.org/shemminger/tcp/2.6.19-rc4/dslThe only obvious problem is Vegas looks broken.
Thanks, Stephen.
It seems that the default Vegas alpha parameter in the rc4 is 1...
I observed similar situation with the NS2Linux simulator (with 2.6.16 code) and found that if alpha=1, delayed ack will make it broken (keeping cwnd very low without real congestion)
See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas
(Basically alpha==1 means Vegas seeks to see a delay of about 1 packet worth. With delayed ack, 1 packet worth of delay is common even with no congestion.)
To make Vegas work, I'd suggest to raise alpha to at least 2 or 3. (and beta has to be at least as large as alpha.)
-David
Signature · 3 lines
Xiaoliang (David) Wei Graduate Student, CS@Caltech http://davidwei.org ***********************************************
Re: TCP congestion graphs (2.6.19-rc4)
Quoted 16 lines · On Tue, 31 Oct 2006 15:25:16 -0800 "Xiaoliang (David) Wei"…
Quoted 4 lines · Stephen Hemminger
I reran the tests, this time using my DSL line to home (80ms 1Mbit/sec).
http://developer.osdl.org/shemminger/tcp/2.6.19-rc4/dslThe only obvious problem is Vegas looks broken.
Thanks, Stephen.
It seems that the default Vegas alpha parameter in the rc4 is 1...
I observed similar situation with the NS2Linux simulator (with 2.6.16 code) and found that if alpha=1, delayed ack will make it broken (keeping cwnd very low without real congestion)
See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas
Quoted 9 lines
(Basically alpha==1 means Vegas seeks to see a delay of about 1 packet worth. With delayed ack, 1 packet worth of delay is common even with no congestion.)
To make Vegas work, I'd suggest to raise alpha to at least 2 or 3. (and beta has to be at least as large as alpha.)
-David
I ran with the current default:
alpha = 1 (scaled 2)
beta = 3 (scaled 6)
gamma = 1 (scaled 2)
Signature · 1 line
Stephen Hemminger <[email protected]>
David Miller
maintainerRe: TCP congestion graphs (2.6.19-rc4)
From: Stephen Hemminger <[email protected]>
Date: Tue, 31 Oct 2006 16:10:07 -0800
Quoted 18 lines
Quoted 15 lines · On Tue, 31 Oct 2006 15:25:16 -0800 "Xiaoliang (David) Wei"…
It seems that the default Vegas alpha parameter in the rc4 is 1...
I observed similar situation with the NS2Linux simulator (with 2.6.16 code) and found that if alpha=1, delayed ack will make it broken (keeping cwnd very low without real congestion)
See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas
(Basically alpha==1 means Vegas seeks to see a delay of about 1 packet worth. With delayed ack, 1 packet worth of delay is common even with no congestion.)
To make Vegas work, I'd suggest to raise alpha to at least 2 or 3. (and beta has to be at least as large as alpha.)
Quoted 7 lines
Quoted 2 lines
-David
I ran with the current default:
alpha = 1 (scaled 2)
beta = 3 (scaled 6)
gamma = 1 (scaled 2)
Testing with alpha=2 and beta=4 would be interesting.
Brakmo and Peterson don't see this delayed ACK behavior in their tests in the original Vegas paper. I read through it a few times and I see no explicit mention of whether they had delayed ACKs enabled or not in any of there tests.
I seem to recall that it was very popular around that time to analyze TCP changes with delayed ACKs disabled since it made the results easier to analyze (which in my opinion invalidates any such results, since in the real world you never run with delayed ACKs off). So perhaps the Vegas folks were doing something similar and just didn't think it was worth mentioning :-)
BTW, Stephen, can you give a short HOWTO on how you generate those graphs, given a tcpdump trace? Thanks a lot.
Re: TCP congestion graphs (2.6.19-rc4)
Quoted 2 lines · David Miller
BTW, Stephen, can you give a short HOWTO on how you generate those graphs, given a tcpdump trace? Thanks a lot.
That would be great.
Please also describe your setup so others algos could be tested too. Thank you.
Signature · 1 line
Evgeniy Polyakov
Re: TCP congestion graphs (2.6.19-rc4)
Quoted 8 lines · On Wed, 1 Nov 2006 08:25:00 +0300 Evgeniy Polyakov <[email protected]> wrote:
Quoted 2 lines · David Miller
BTW, Stephen, can you give a short HOWTO on how you generate those graphs, given a tcpdump trace? Thanks a lot.
That would be great.
Please also describe your setup so others algos could be tested too. Thank you.
Some basic stuff is on the parent page
http://developer.osdl.org/shemminger/tcp
David Miller
maintainerRe: TCP congestion graphs (2.6.19-rc4)
From: Stephen Hemminger <[email protected]>
Date: Tue, 31 Oct 2006 21:58:48 -0800
Quoted 2 lines
Some basic stuff is on the parent page
http://developer.osdl.org/shemminger/tcp
Are you using tcptrace to generate those plots? If so are you using xplot or gnuplot to build those image files?
Thanks.
Re: TCP congestion graphs (2.6.19-rc4)
Quoted 10 lines · On Wed, 01 Nov 2006 21:44:18 -0800 (PST) David Miller <[email protected]>…
From: Stephen Hemminger <[email protected]>
Date: Tue, 31 Oct 2006 21:58:48 -0800Quoted 2 lines
Some basic stuff is on the parent page
http://developer.osdl.org/shemminger/tcpAre you using tcptrace to generate those plots? If so are you using xplot or gnuplot to build those image files?
Description now up on linux-net
http://linux-net.osdl.org/index.php/TCP_testing
I use tcpprobe kernel module because you can't directly see congestion window from packet captures.
Signature · 1 line
Stephen Hemminger <[email protected]>
David Miller
maintainerRe: TCP congestion graphs (2.6.19-rc4)
In reply to David MillerFrom: David Miller <[email protected]>
Date: Tue, 31 Oct 2006 17:30:28 -0800 (PST)
Quoted 24 lines
From: Stephen Hemminger <[email protected]>
Date: Tue, 31 Oct 2006 16:10:07 -0800Quoted 21 lines
Quoted 17 lines · On Tue, 31 Oct 2006 15:25:16 -0800 "Xiaoliang (David) Wei"…
It seems that the default Vegas alpha parameter in the rc4 is 1...
I observed similar situation with the NS2Linux simulator (with 2.6.16 code) and found that if alpha=1, delayed ack will make it broken (keeping cwnd very low without real congestion)
See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas
(Basically alpha==1 means Vegas seeks to see a delay of about 1 packet worth. With delayed ack, 1 packet worth of delay is common even with no congestion.)
To make Vegas work, I'd suggest to raise alpha to at least 2 or 3. (and beta has to be at least as large as alpha.)
-David
Quoted 6 lines
Quoted 4 lines
I ran with the current default:
alpha = 1 (scaled 2)
beta = 3 (scaled 6)
gamma = 1 (scaled 2)Testing with alpha=2 and beta=4 would be interesting.
Instead of letting this issue rot, I've checked the following into net-2.6.20
commit cd7f265b9069d8fd66a33d37139821f84ef04f0e
Author: David S. Miller <[email protected]> Date: Tue Nov 28 14:37:38 2006 -0800
[TCP] Vegas: Increase default alpha to 2 and beta to 4.
This helps Vegas cope better with delayed ACKs, see analysis at:
http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas
net/ipv4/tcp_vegas.c +2 -2
@@ -42,8 +42,8 @@ #include <net/tcp.h>* with V_PARAM_SHIFT bits to the right of the binary point. */ #define V_PARAM_SHIFT 1 -static int alpha =1 <<V_PARAM_SHIFT ; -static int beta =3 <<V_PARAM_SHIFT ; +static int alpha =2 <<V_PARAM_SHIFT ; +static int beta =4 <<V_PARAM_SHIFT ;static int gamma =1 <<V_PARAM_SHIFT ;module_param (alpha, int,0644 );