TCP performance drop 2.6.6->2.6.7

DISCUSSION Baruch Even 4 participants 7 messages Apr 12, 2005 – Apr 15, 2005

Subsystems (2)

Networking Network drivers

lore

Share & tools

lore

Baruch Even

author

2005-04-12 22:18 UTC #0 lore raw

Hello,

I'm trying to port my H-TCP and SACK improvements to 2.6.11, but I seem to hit performance problems that are unrelated to what I worked on so far.

My tests show that between 2.6.6 and 2.6.7 the TCP performance dropped considerably.

The setup for the tests, vanilla kernels, dummynet with 100 Mbit/s and 40ms rtt using the BIC protocol. No patches applied whatsoever.

iperf tests for 2.6.6 get about 90Mbit/s while 2.6.7 gets 30Mbit/s.

I was wondering if someone can think of a reason why this happens?

Is there a way to get the different network related patches between these two versions? I don't have access to bk to get it myself.

Baruch

David S. Miller

maintainer

2005-04-12 22:47 UTC #1 lore raw

Quoted 1 line · On Tue, 12 Apr 2005 23:18:35 +0100 Baruch Even <[email protected]> wrote:

I was wondering if someone can think of a reason why this happens?

Probably some bug we fixed in 2.6.8 or later...

Quoted 2 lines

Is there a way to get the different network related patches between these two versions? I don't have access to bk to get it myself.

Get patch-2.6.7.bz2 or whatever from kernel.org, then scan around for changes to include/*/tcp.h and net/ipv4/tcp*.c

Stephen Hemminger

2005-04-12 23:07 UTC #2 lore raw

In reply to Baruch Even

Quoted 7 lines · On Tue, 12 Apr 2005 23:18:35 +0100 Baruch Even <[email protected]> wrote:

Hello,
I'm trying to port my H-TCP and SACK improvements to 2.6.11, but I seem to hit performance problems that are unrelated to what I worked on so far.
My tests show that between 2.6.6 and 2.6.7 the TCP performance dropped considerably.

Remember 2.6.6 had the old hyper aggressive and over correcting version of BIC.

Quoted 4 lines

The setup for the tests, vanilla kernels, dummynet with 100 Mbit/s and 40ms rtt using the BIC protocol. No patches applied whatsoever.
iperf tests for 2.6.6 get about 90Mbit/s while 2.6.7 gets 30Mbit/s.

Haven't measured old kernels recently. But haven't seen that with the newest stuff I tested last night. At 40ms delay with netem over 1G to 100M bridge.

	Reno	99.6
	Vegas	61.6
	BIC	  97.8
	Hstcp	 99.4
	Westwood 99.4

What are your sysctl settings.

Quoted 6 lines

I was wondering if someone can think of a reason why this happens?
Is there a way to get the different network related patches between these two versions? I don't have access to bk to get it myself.
Baruch

Baruch Even

author

2005-04-12 23:59 UTC #3 lore raw

Quoted 7 lines · Stephen Hemminger

Quoted 3 lines · On Tue, 12 Apr 2005 23:18:35 +0100 Baruch Even <[email protected]> wrote:
The setup for the tests, vanilla kernels, dummynet with 100 Mbit/s and 40ms rtt using the BIC protocol. No patches applied whatsoever.
iperf tests for 2.6.6 get about 90Mbit/s while 2.6.7 gets 30Mbit/s.

Quoted 8 lines

Haven't measured old kernels recently. But haven't seen that with the newest stuff I tested last night. At 40ms delay with netem over 1G to 100M bridge.
	Reno	99.6
	Vegas	61.6
	BIC	  97.8
	Hstcp	 99.4
	Westwood 99.4

I've now tested 2.6.11.7 and I get about 50Mbit/s, it starts at about 100 and drops to about 45. Tested for about a minute but it didn't show signs of recovery.

Quoted 1 line

What are your sysctl settings.

Attached.

Baruch

icmp_echo_ignore_all 0
icmp_echo_ignore_broadcasts 0
icmp_ignore_bogus_error_responses 0
icmp_ratelimit 1000
icmp_ratemask 6168
igmp_max_memberships 20
igmp_max_msf 10
inet_peer_gc_maxtime 120
inet_peer_gc_mintime 10
inet_peer_maxttl 600
inet_peer_minttl 120
inet_peer_threshold 65664
ip_autoconfig 0
ip_default_ttl 64
ip_dynaddr 0
ip_forward 0
ip_local_port_range 32768 61000
ip_no_pmtu_disc 0
ip_nonlocal_bind 0
ipfrag_high_thresh 262144
ipfrag_low_thresh 196608
ipfrag_secret_interval 600
ipfrag_time 30
tcp_abort_on_overflow 0
tcp_adv_win_scale 2
tcp_app_win 31
tcp_bic 1
tcp_bic_beta 819
tcp_bic_fast_convergence 1
tcp_bic_low_window 14
tcp_dsack 1
tcp_ecn 0
tcp_fack 1
tcp_fin_timeout 60
tcp_frto 0
tcp_keepalive_intvl 75
tcp_keepalive_probes 9
tcp_keepalive_time 7200
tcp_low_latency 0
tcp_max_orphans 8192
tcp_max_syn_backlog 1024
tcp_max_tw_buckets 180000
tcp_mem 8388608 8388608 8388608
tcp_moderate_rcvbuf 1
tcp_no_metrics_save 0
tcp_orphan_retries 0
tcp_reordering 3
tcp_retrans_collapse 1
tcp_retries1 3
tcp_retries2 15
tcp_rfc1337 0
tcp_rmem 4096 87380 8388608
tcp_sack 1
tcp_stdurg 0
tcp_syn_retries 5
tcp_synack_retries 5
tcp_timestamps 1
tcp_tso_win_divisor 8
tcp_tw_recycle 0
tcp_tw_reuse 0
tcp_vegas_alpha 2
tcp_vegas_beta 6
tcp_vegas_cong_avoid 0
tcp_vegas_gamma 2
tcp_westwood 0
tcp_window_scaling 1
tcp_wmem 4096 87380 8388608

Stephen Hemminger

2005-04-13 18:20 UTC #4 lore raw

Quoted 5 lines · On Wed, 13 Apr 2005 00:59:25 +0100 Baruch Even <[email protected]> wrote:

Quoted 4 lines · Stephen Hemminger
Quoted 2 lines · On Tue, 12 Apr 2005 23:18:35 +0100 Baruch Even <[email protected]> wrote:
The setup for the tests, vanilla kernels, dummynet with 100 Mbit/s and 40ms rtt using the BIC protocol. No patches applied whatsoever.

What is the queue length? Better yet what is the exact commandline used to setup dummynet.

Quoted 22 lines

Quoted 4 lines
Quoted 2 lines
iperf tests for 2.6.6 get about 90Mbit/s while 2.6.7 gets 30Mbit/s.
Quoted 8 lines
Haven't measured old kernels recently. But haven't seen that with the newest stuff I tested last night. At 40ms delay with netem over 1G to 100M bridge.
	Reno	99.6
	Vegas	61.6
	BIC	  97.8
	Hstcp	 99.4
	Westwood 99.4
I've now tested 2.6.11.7 and I get about 50Mbit/s, it starts at about 100 and drops to about 45. Tested for about a minute but it didn't show signs of recovery.
Quoted 1 line
What are your sysctl settings.
Attached.
Baruch

Baruch Even

author

2005-04-15 14:37 UTC #5 lore raw

In reply to Baruch Even

I have no clue *why* it happens, but a hint on when it happens. If I remove the ixgb driver from my .config, performance is back to normal. Note though that I don't even use the ixgb driver for my tests, it's just sitting idle on the machine and connected to nothing (the cisco router is turned off to reduce noise polution).

To continue my work I'll disable the ixgb for now.

Baruch

Quoted 16 lines · Baruch Even

Hello,
I'm trying to port my H-TCP and SACK improvements to 2.6.11, but I seem to hit performance problems that are unrelated to what I worked on so far.
My tests show that between 2.6.6 and 2.6.7 the TCP performance dropped considerably.
The setup for the tests, vanilla kernels, dummynet with 100 Mbit/s and 40ms rtt using the BIC protocol. No patches applied whatsoever.
iperf tests for 2.6.6 get about 90Mbit/s while 2.6.7 gets 30Mbit/s.
I was wondering if someone can think of a reason why this happens?
Is there a way to get the different network related patches between these two versions? I don't have access to bk to get it myself.
Baruch

Nivedita Singhvi

2005-04-15 15:30 UTC #6 lore raw

Quoted 1 line · Baruch Even

I have no clue *why* it happens, but a hint on when it happens. If I remove the ixgb driver from my .config, performance is back to normal. Note though that I don't even use the ixgb driver for my tests, it's just sitting idle on the machine and connected to nothing (the cisco router is turned off to reduce noise polution).

Check your interrupts - it's probably sending out a large number of interrupts which are clogging the system..

thanks,
Nivedita