Skip to contentSkip to conversation

TCP congestion graphs

DISCUSSION Stephen Hemminger 6 participants 14 messages Oct 25, 2006 – Nov 28, 2006

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

I ran some congestion window tests against 2.6.19-rc3. For congestion window graphs see:

	http://developer.osdl.org/shemminger/tcp/2.6.19-rc3/

The connection was a single flow with a 500ms RTT and a 100Mbit slowest link speed.

BIC OK
CUBIC OK (after patch)
HIGHSPEED BROKEN?
HTCP OK
RENO OK (massive overshoot)
SCALEABLE OK
VEGAS OK (no better than Reno)
VENO Same as Reno?
WESTWOOD Overshoot then low window

Highspeed has something wrong (test glitch?) Westwood behaves poorly
Veno seems no better than either Reno or Vegas in this case.

The end systems were from latest git. The netem bridge is using 2.6.18-rt with a patch to netem to use hrtimers. iperf was modified to allow easy selection of congestion control.

Signature · 1 line
Stephen Hemminger <[email protected]>
#1 lore raw

Not sure why the slow start for cubic is slower than the others. We will check on this.

----- Original Message ----- From: "Stephen Hemminger" <[email protected]>
To: "Douglas Leith" <[email protected]>; "Sangtae Ha" <[email protected]>
Cc: <[email protected]>
Sent: Wednesday, October 25, 2006 2:02 PM
Subject: TCP congestion graphs

Quoted 32 lines

I ran some congestion window tests against 2.6.19-rc3. For congestion window graphs see:
http://developer.osdl.org/shemminger/tcp/2.6.19-rc3/

The connection was a single flow with a 500ms RTT and a 100Mbit slowest link speed.

BIC OK
CUBIC OK (after patch)
HIGHSPEED BROKEN?
HTCP OK
RENO OK (massive overshoot)
SCALEABLE OK
VEGAS OK (no better than Reno)
VENO Same as Reno?
WESTWOOD Overshoot then low window

Highspeed has something wrong (test glitch?) Westwood behaves poorly
Veno seems no better than either Reno or Vegas in this case.

The end systems were from latest git. The netem bridge is using 2.6.18-rt with a patch to netem to use hrtimers. iperf was modified to allow easy selection of congestion control.

Signature · 5 lines
Stephen Hemminger <[email protected]>
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#2 lore raw
Quoted 3 lines · On Wed, 25 Oct 2006 18:34:17 -0400 "Injong Rhee" <[email protected]> wrote:

Not sure why the slow start for cubic is slower than the others. We will check on this.

I think it is because cubic initializes with a ssthresh of 100, and others leave ssthresh uninitialized until the first loss. This causes cubic to change to conservative mode once it crosses the initial ssthresh. Other protocols just keep doing additive increase.

My large BDP test has a 500ms delay (250ms each way) and a real 100mbit link on the last hop.
Because the intermediate emulator has a large queue (10,000 packets) large growth is possible before the first loss event. There were also no other flows during this test to fill the intermediate queue.

This may very well be a bogus test, I just was looking for large BDP behavior quirks. Any test without background traffic as you have argued is unrealistic.

#3 lore raw

Hi Stephen,

is your rt-patch to netem public available?

Best regards

HGN

Signature · 3 lines
Signed and/or encrypted mails preferd. Key-Id = 0x98350C22
Fingerprint = 490F 557B 6C48 6D7E 5706  2EA2 4A22 8D45 9835 0C22 
Key available under: www.jauu.net/download/gnupg_key 
#6 lore raw

Re: TCP congestion graphs (2.6.19-rc4)

Quoted 4 lines · Stephen Hemminger

I reran the tests, this time using my DSL line to home (80ms 1Mbit/sec).

        http://developer.osdl.org/shemminger/tcp/2.6.19-rc4/dsl

The only obvious problem is Vegas looks broken.

Thanks, Stephen.

It seems that the default Vegas alpha parameter in the rc4 is 1...

I observed similar situation with the NS2Linux simulator (with 2.6.16 code) and found that if alpha=1, delayed ack will make it broken (keeping cwnd very low without real congestion)

See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas

(Basically alpha==1 means Vegas seeks to see a delay of about 1 packet worth. With delayed ack, 1 packet worth of delay is common even with no congestion.)

To make Vegas work, I'd suggest to raise alpha to at least 2 or 3. (and beta has to be at least as large as alpha.)

-David

Signature · 3 lines
Xiaoliang (David) Wei      Graduate Student, CS@Caltech
http://davidwei.org
***********************************************
#7 lore raw

Re: TCP congestion graphs (2.6.19-rc4)

Quoted 16 lines · On Tue, 31 Oct 2006 15:25:16 -0800 "Xiaoliang (David) Wei"…
Quoted 4 lines · Stephen Hemminger

I reran the tests, this time using my DSL line to home (80ms 1Mbit/sec).

        http://developer.osdl.org/shemminger/tcp/2.6.19-rc4/dsl

The only obvious problem is Vegas looks broken.

Thanks, Stephen.

It seems that the default Vegas alpha parameter in the rc4 is 1...

I observed similar situation with the NS2Linux simulator (with 2.6.16 code) and found that if alpha=1, delayed ack will make it broken (keeping cwnd very low without real congestion)

See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas

Quoted 9 lines

(Basically alpha==1 means Vegas seeks to see a delay of about 1 packet worth. With delayed ack, 1 packet worth of delay is common even with no congestion.)

To make Vegas work, I'd suggest to raise alpha to at least 2 or 3. (and beta has to be at least as large as alpha.)

-David

I ran with the current default:
alpha = 1 (scaled 2)
beta = 3 (scaled 6)
gamma = 1 (scaled 2)

Signature · 1 line
Stephen Hemminger <[email protected]>
maintainer
#8 lore raw

Re: TCP congestion graphs (2.6.19-rc4)

From: Stephen Hemminger <[email protected]>
Date: Tue, 31 Oct 2006 16:10:07 -0800

Quoted 18 lines
Quoted 15 lines · On Tue, 31 Oct 2006 15:25:16 -0800 "Xiaoliang (David) Wei"…

It seems that the default Vegas alpha parameter in the rc4 is 1...

I observed similar situation with the NS2Linux simulator (with 2.6.16 code) and found that if alpha=1, delayed ack will make it broken (keeping cwnd very low without real congestion)

See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas

(Basically alpha==1 means Vegas seeks to see a delay of about 1 packet worth. With delayed ack, 1 packet worth of delay is common even with no congestion.)

To make Vegas work, I'd suggest to raise alpha to at least 2 or 3. (and beta has to be at least as large as alpha.)

Quoted 7 lines
Quoted 2 lines

-David

I ran with the current default:
alpha = 1 (scaled 2)
beta = 3 (scaled 6)
gamma = 1 (scaled 2)

Testing with alpha=2 and beta=4 would be interesting.

Brakmo and Peterson don't see this delayed ACK behavior in their tests in the original Vegas paper. I read through it a few times and I see no explicit mention of whether they had delayed ACKs enabled or not in any of there tests.

I seem to recall that it was very popular around that time to analyze TCP changes with delayed ACKs disabled since it made the results easier to analyze (which in my opinion invalidates any such results, since in the real world you never run with delayed ACKs off). So perhaps the Vegas folks were doing something similar and just didn't think it was worth mentioning :-)

BTW, Stephen, can you give a short HOWTO on how you generate those graphs, given a tcpdump trace? Thanks a lot.

#9 lore raw

Re: TCP congestion graphs (2.6.19-rc4)

Quoted 2 lines · David Miller

BTW, Stephen, can you give a short HOWTO on how you generate those graphs, given a tcpdump trace? Thanks a lot.

That would be great.
Please also describe your setup so others algos could be tested too. Thank you.

Signature · 1 line
	Evgeniy Polyakov
#10 lore raw

Re: TCP congestion graphs (2.6.19-rc4)

Quoted 8 lines · On Wed, 1 Nov 2006 08:25:00 +0300 Evgeniy Polyakov <[email protected]> wrote:
Quoted 2 lines · David Miller

BTW, Stephen, can you give a short HOWTO on how you generate those graphs, given a tcpdump trace? Thanks a lot.

That would be great.
Please also describe your setup so others algos could be tested too. Thank you.

Some basic stuff is on the parent page
http://developer.osdl.org/shemminger/tcp

#12 lore raw

Re: TCP congestion graphs (2.6.19-rc4)

Quoted 10 lines · On Wed, 01 Nov 2006 21:44:18 -0800 (PST) David Miller <[email protected]>…

From: Stephen Hemminger <[email protected]>
Date: Tue, 31 Oct 2006 21:58:48 -0800

Quoted 2 lines

Some basic stuff is on the parent page
http://developer.osdl.org/shemminger/tcp

Are you using tcptrace to generate those plots? If so are you using xplot or gnuplot to build those image files?

Description now up on linux-net
http://linux-net.osdl.org/index.php/TCP_testing

I use tcpprobe kernel module because you can't directly see congestion window from packet captures.

Signature · 1 line
Stephen Hemminger <[email protected]>
maintainer
#13 lore raw

Re: TCP congestion graphs (2.6.19-rc4)

In reply to David Miller

From: David Miller <[email protected]>
Date: Tue, 31 Oct 2006 17:30:28 -0800 (PST)

Quoted 24 lines

From: Stephen Hemminger <[email protected]>
Date: Tue, 31 Oct 2006 16:10:07 -0800

Quoted 21 lines
Quoted 17 lines · On Tue, 31 Oct 2006 15:25:16 -0800 "Xiaoliang (David) Wei"…

It seems that the default Vegas alpha parameter in the rc4 is 1...

I observed similar situation with the NS2Linux simulator (with 2.6.16 code) and found that if alpha=1, delayed ack will make it broken (keeping cwnd very low without real congestion)

See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas

(Basically alpha==1 means Vegas seeks to see a delay of about 1 packet worth. With delayed ack, 1 packet worth of delay is common even with no congestion.)

To make Vegas work, I'd suggest to raise alpha to at least 2 or 3. (and beta has to be at least as large as alpha.)

-David

Quoted 6 lines
Quoted 4 lines

I ran with the current default:
alpha = 1 (scaled 2)
beta = 3 (scaled 6)
gamma = 1 (scaled 2)

Testing with alpha=2 and beta=4 would be interesting.

Instead of letting this issue rot, I've checked the following into net-2.6.20

commit cd7f265b9069d8fd66a33d37139821f84ef04f0e

Author: David S. Miller <[email protected]>
Date:   Tue Nov 28 14:37:38 2006 -0800

[TCP] Vegas: Increase default alpha to 2 and beta to 4.

This helps Vegas cope better with delayed ACKs, see analysis at:

http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas

net/ipv4/tcp_vegas.c +2 -2
index a3b7aa0..ddc4bcc 100644
@@ -42,8 +42,8 @@ #include <net/tcp.h>
  * with V_PARAM_SHIFT bits to the right of the binary point.
  */
 #define V_PARAM_SHIFT 1
-static int alpha = 1<<V_PARAM_SHIFT;
-static int beta  = 3<<V_PARAM_SHIFT;
+static int alpha = 2<<V_PARAM_SHIFT;
+static int beta  = 4<<V_PARAM_SHIFT;
 static int gamma = 1<<V_PARAM_SHIFT;
 
 module_param(alpha, int, 0644);