Skip to contentSkip to conversation

2.6.18-rc[67] crashes in TCP ack handling

PATCH In review bert hubert 6 participants 40 messages Sep 16, 2006 – Sep 25, 2006 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

The bad news is that I haven't yet been able to capture traces. Once every three days or so I get a crash of 2.6.18-rc[67] which *probably* end in tcp_ack(), but I don't have the exact dump.

My .config is indeed heavy on TCP congestion stuff:

$ zcat /proc/config.gz | grep -i tcp
CONFIG_INET_TCP_DIAG=y
CONFIG_TCP_CONG_ADVANCED=y
# TCP congestion control
CONFIG_TCP_CONG_BIC=y
CONFIG_TCP_CONG_CUBIC=y
CONFIG_TCP_CONG_WESTWOOD=y
CONFIG_TCP_CONG_HTCP=y
CONFIG_TCP_CONG_HSTCP=y
CONFIG_TCP_CONG_HYBLA=y
CONFIG_TCP_CONG_VEGAS=y
CONFIG_TCP_CONG_SCALABLE=y
CONFIG_TCP_CONG_LP=y
CONFIG_TCP_CONG_VENO=y
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
CONFIG_IP_NF_TARGET_TCPMSS=y
# CONFIG_NET_TCPPROBE is not set
# CONFIG_ISCSI_TCP is not set
# CONFIG_NFSD_TCP is not set

However, I haven't specifically configured anything.

$ dmesg | grep -i tcp 
[   33.106317] TCP established hash table entries: 131072 (order: 8, 1048576
bytes)
[   33.107086] TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
[   33.107476] TCP: Hash tables configured (established 131072 bind 65536)
[   33.107605] TCP reno registered
[   40.985770] IPVS: Registered protocols (TCP, UDP, AH, ESP)
[   41.105710] TCP bic registered
[   41.105833] TCP cubic registered
[   41.105957] TCP westwood registered
[   41.106080] TCP highspeed registered
[   41.106203] TCP hybla registered
[   41.106328] TCP htcp registered
[   41.106452] TCP vegas registered
[   41.106574] TCP veno registered
[   41.106698] TCP scalable registered
[   41.106822] TCP lp registered
$ cat ipv4/tcp_congestion_control
lp

I hope to follow up this message with the actual backtrace, but this is already an heads up.

Sorry for not yet being able to be more specific.

bert

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
#1 lore raw
Quoted 49 lines · On Sun, 17 Sep 2006 00:19:48 +0200 bert hubert <[email protected]>…

The bad news is that I haven't yet been able to capture traces. Once every three days or so I get a crash of 2.6.18-rc[67] which *probably* end in tcp_ack(), but I don't have the exact dump.

My .config is indeed heavy on TCP congestion stuff:

$ zcat /proc/config.gz | grep -i tcp
CONFIG_INET_TCP_DIAG=y
CONFIG_TCP_CONG_ADVANCED=y
# TCP congestion control
CONFIG_TCP_CONG_BIC=y
CONFIG_TCP_CONG_CUBIC=y
CONFIG_TCP_CONG_WESTWOOD=y
CONFIG_TCP_CONG_HTCP=y
CONFIG_TCP_CONG_HSTCP=y
CONFIG_TCP_CONG_HYBLA=y
CONFIG_TCP_CONG_VEGAS=y
CONFIG_TCP_CONG_SCALABLE=y
CONFIG_TCP_CONG_LP=y
CONFIG_TCP_CONG_VENO=y
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
CONFIG_IP_NF_TARGET_TCPMSS=y
# CONFIG_NET_TCPPROBE is not set
# CONFIG_ISCSI_TCP is not set
# CONFIG_NFSD_TCP is not set

However, I haven't specifically configured anything.

$ dmesg | grep -i tcp 
[   33.106317] TCP established hash table entries: 131072 (order: 8, 1048576
bytes)
[   33.107086] TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
[   33.107476] TCP: Hash tables configured (established 131072 bind 65536)
[   33.107605] TCP reno registered
[   40.985770] IPVS: Registered protocols (TCP, UDP, AH, ESP)
[   41.105710] TCP bic registered
[   41.105833] TCP cubic registered
[   41.105957] TCP westwood registered
[   41.106080] TCP highspeed registered
[   41.106203] TCP hybla registered
[   41.106328] TCP htcp registered
[   41.106452] TCP vegas registered
[   41.106574] TCP veno registered
[   41.106698] TCP scalable registered
[   41.106822] TCP lp registered
$ cat ipv4/tcp_congestion_control
lp
Quoted 6 lines

I hope to follow up this message with the actual backtrace, but this is already an heads up.

Sorry for not yet being able to be more specific.

bert

By building all the possiblities into the kernel, ie. not as modules you get the last one registered. TCP LP is probably the worst one to use, because it is designed for bulk low priority applications. It also is one of the newest least tested. Right now, I would rate the choices as:
Stable: reno, bic, htcp, westwood, cubic
Testing: hybla, highspeed, scaleable, vegas
Unstable: veno, lp

The stable ones seem to be reliable and have no known issues. The testing group are only interesting for comparison, and need more tuning. The unstable group, are new and need even more testing for fairness and robustness.

Without a back trace, it will be hard to find the bug in TCP LP

#2 lore raw
Quoted 4 lines · Stephen Hemminger

By building all the possiblities into the kernel, ie. not as modules you get the last one registered. TCP LP is probably the worst one to use, because it is designed for bulk low priority applications. It also is one of the newest least tested. Right now, I would rate

Hehe, this seems to be a bad default configuration policy then. People generally don't assume that if the kernel offers 10 policies that the most unstable will be used by default if you compile them all in.

I've attached a patch that reorders the choices per your suggested order, so people are most likely to get a sane default.

I've tried to make "reno" the default, no matter what you compiled in, but it didn't work. The linker probably reorders tcp_cong.o in early.

Quoted 1 line

Without a back trace, it will be hard to find the bug in TCP LP

Indeed.

Many thanks for your quick answer Stephen!

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
net/ipv4/Makefile renamed from net/ipv4/Makefile~ +12 -8
@@ -7,7 +7,7 @@
 	     ip_output.o ip_sockglue.o inet_hashtables.o \
 	     inet_timewait_sock.o inet_connection_sock.o \
 	     tcp.o tcp_input.o tcp_output.o tcp_timer.o tcp_ipv4.o \
-	     tcp_minisocks.o tcp_cong.o \
+	     tcp_minisocks.o \
 	     datagram.o raw.o udp.o arp.o icmp.o devinet.o af_inet.o igmp.o \
 	     sysctl_net_ipv4.o fib_frontend.o fib_semantics.o
 
@@ -37,16 +37,20 @@
 obj-$(CONFIG_IP_ROUTE_MULTIPATH_CACHED) += multipath.o
 obj-$(CONFIG_INET_TCP_DIAG) += tcp_diag.o
 obj-$(CONFIG_NET_TCPPROBE) += tcp_probe.o
-obj-$(CONFIG_TCP_CONG_BIC) += tcp_bic.o
-obj-$(CONFIG_TCP_CONG_CUBIC) += tcp_cubic.o
-obj-$(CONFIG_TCP_CONG_WESTWOOD) += tcp_westwood.o
+
+obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o
+obj-$(CONFIG_TCP_CONG_VENO) += tcp_veno.o
+obj-$(CONFIG_TCP_CONG_VEGAS) += tcp_vegas.o
+obj-$(CONFIG_TCP_CONG_SCALABLE) += tcp_scalable.o
 obj-$(CONFIG_TCP_CONG_HSTCP) += tcp_highspeed.o
 obj-$(CONFIG_TCP_CONG_HYBLA) += tcp_hybla.o
+obj-$(CONFIG_TCP_CONG_CUBIC) += tcp_cubic.o
+obj-$(CONFIG_TCP_CONG_WESTWOOD) += tcp_westwood.o
 obj-$(CONFIG_TCP_CONG_HTCP) += tcp_htcp.o
-obj-$(CONFIG_TCP_CONG_VEGAS) += tcp_vegas.o
-obj-$(CONFIG_TCP_CONG_VENO) += tcp_veno.o
-obj-$(CONFIG_TCP_CONG_SCALABLE) += tcp_scalable.o
-obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o
+obj-$(CONFIG_TCP_CONG_BIC) += tcp_bic.o
+
+# make sure the built in congestion scheme is the default
+obj-y += tcp_cong.o
 
 obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \
 		      xfrm4_output.o
#3 lore raw
Quoted 11 lines · On Sun, 17 Sep 2006 12:11:01 +0200 bert hubert <[email protected]>…
Quoted 4 lines · Stephen Hemminger

By building all the possiblities into the kernel, ie. not as modules you get the last one registered. TCP LP is probably the worst one to use, because it is designed for bulk low priority applications. It also is one of the newest least tested. Right now, I would rate

Hehe, this seems to be a bad default configuration policy then. People generally don't assume that if the kernel offers 10 policies that the most unstable will be used by default if you compile them all in.

Quoted 5 lines

I've attached a patch that reorders the choices per your suggested order, so people are most likely to get a sane default.

I've tried to make "reno" the default, no matter what you compiled in, but it didn't work. The linker probably reorders tcp_cong.o in early.

Reno is always builtin. To choose it use
sysctl -w net.ipv4.tcp_congestion_control=reno
Depending on load order is not good, and not a safe way to configure. If you want a particular value set it with sysctl!

The more experimental ones are already labeled as EXPERIMENTAL.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#5 lore raw

Re: tcp congestion policy selection link order fragile

Quoted 11 lines · On Sun, 17 Sep 2006 14:21:53 +0200 bert hubert <[email protected]>…
Quoted 1 line · Stephen Hemminger

Depending on load order is not good, and not a safe way to configure.

I agree fully.

Quoted 1 line

If you want a particular value set it with sysctl!

Operators, distributors and even people who've been doing kernel stuff for more than a decade expect to be able to compile in (experimental) policies, and not have a *random* one of them enabled by default!

Any body who builds in random stuff without thinking is being foolish. But, if you can think of a better configuration method that isn't too grotty, then go for it.

#6 lore raw

Re: tcp congestion policy selection link order fragile

The original message Stephen reacts to below apparently never made it to the list, it can be found here: http://ds9a.nl/tmp/module-policy.txt

Quoted 3 lines

Any body who builds in random stuff without thinking is being foolish. But, if you can think of a better configuration method that isn't too grotty, then go for it.

The method I'm proposing is simple enough:

1) reno is always built-in
2) it is the default tcp congestion policy
3) loading/compiling-in additional tcp congestion policies only make them available
4) userspace is free to select a non-default tcp congestion policy at will

The implementation might be as simple as making the *first* registered congestion policy the default (instead of the last one) which would be reno, as it is in tcp_cong.o, which is probably always loaded first (as the other .o's need symbols that are in tcp_cong.o).

Despite what you allege about my foolishness, I maintain that a kernel that enables a *random policy* from the ones you compiled in, is not a sane kernel.

The default kernel should be as sane as possible, allowing the userspace people (ie, me) to mess things up to their heart's desire.

Bert

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
#7 lore raw

Re: tcp congestion policy selection link order fragile

Quoted 11 lines · On Sun, 17 Sep 2006 16:51:50 +0200 bert hubert <[email protected]>…

The original message Stephen reacts to below apparently never made it to the list, it can be found here: http://ds9a.nl/tmp/module-policy.txt

Quoted 3 lines

Any body who builds in random stuff without thinking is being foolish. But, if you can think of a better configuration method that isn't too grotty, then go for it.

The method I'm proposing is simple enough:

1) reno is always built-in
2) it is the default tcp congestion policy

No, Reno is unstable in high BDP

Quoted 18 lines

3) loading/compiling-in additional tcp congestion policies only make them available
4) userspace is free to select a non-default tcp congestion policy at will

The implementation might be as simple as making the *first* registered congestion policy the default (instead of the last one) which would be reno, as it is in tcp_cong.o, which is probably always loaded first (as the other .o's need symbols that are in tcp_cong.o).

Despite what you allege about my foolishness, I maintain that a kernel that enables a *random policy* from the ones you compiled in, is not a sane kernel.

The default kernel should be as sane as possible, allowing the userspace people (ie, me) to mess things up to their heart's desire.

Bert

maintainer
#8 lore raw

Re: tcp congestion policy selection link order fragile

In reply to a message not in the archive

From: bert hubert <[email protected]>
Date: Sun, 17 Sep 2006 14:21:53 +0200

Quoted 3 lines

Operators, distributors and even people who've been doing kernel stuff for more than a decade expect to be able to compile in (experimental) policies, and not have a *random* one of them enabled by default!

We created TCP_CONG_ADVANCED for a purpose. If you turn that thing on, you get full control but if something breaks you get to keep the pieces.

Quite frankly, just about everyone should not enable TCP_CONG_ADVANCED at all. And quite likely thie applies even distribution vendors.

#9 lore raw

Re: tcp congestion policy selection link order fragile

Quoted 3 lines · David Miller

We created TCP_CONG_ADVANCED for a purpose. If you turn that thing on, you get full control but if something breaks you get to keep the pieces.

But we should not try to break stuff on purpose, no matter how advanced. It makes zero sense. To reiterate, when compiling in multiple TCP policies, a *random* one gets enabled. This is not something we want to offer even advanced users. It is a kernel, not an adventure course.

Please consider this near-oneliner patch which makes stuff behave more like people expect: loading a module, or compiling in a congestion avoidance policy only makes it available, but does not turn it on by default.

It also cleans up two notices a bit.

I've tested this patch and it does the job for me, reno is now the default, even when more advanced options are compiled in, but the rest is still available.

When in doubt, consider that I discovered this because my kernel was crashing, and that this is bound to generate heaps of annoying email otherwise.

Thanks.

net/ipv4/tcp_cong.c renamed from net/ipv4/tcp_cong.c.org +3 -3
@@ -45,11 +45,11 @@
 
 	spin_lock(&tcp_cong_list_lock);
 	if (tcp_ca_find(ca->name)) {
-		printk(KERN_NOTICE "TCP %s already registered\n", ca->name);
+		printk(KERN_NOTICE "TCP congestion control '%s' already registered\n", ca->name);
 		ret = -EEXIST;
 	} else {
-		list_add_rcu(&ca->list, &tcp_cong_list);
-		printk(KERN_INFO "TCP %s registered\n", ca->name);
+		list_add_tail_rcu(&ca->list, &tcp_cong_list);
+		printk(KERN_INFO "TCP congestion control '%s' registered\n", ca->name);
 	}
 	spin_unlock(&tcp_cong_list_lock);
 
Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
maintainer
#10 lore raw

Re: tcp congestion policy selection link order fragile

From: bert hubert <[email protected]>
Date: Mon, 18 Sep 2006 11:59:36 +0200

Quoted 3 lines

I've tested this patch and it does the job for me, reno is now the default, even when more advanced options are compiled in, but the rest is still available.

This breaks our intention that when TCP_CONG_ADVANCED is not set, BIC is the default since that is the default congestion control algorithm we want users to get.

When TCP_CONG_ADVANCED is disabled, we turn on TCP_CONG_BIC, and your changes cause reno to be the default algorithm in that build case. That's not what we want.

Any ordering scheme is wrong or unexpected for _somebody_. Look how easy it was for you to break the BIC default we had in place. To make things sensible for you, your patch causes everyone else got the wrong default. Therefore any ordering scheme is by definition arbitrary and no ordering is better than any other one.

#11 lore raw

Re: tcp congestion policy selection link order fragile

Quoted 1 line · David Miller

Any ordering scheme is wrong or unexpected for _somebody_. Look how

I agree violently. Would you agree that it would be best to have a mechanism that explicitly sets a sane default, and does not rely on ordering?

My implementation indeed broke your intentions, but would you be open to revamping things so the default policy is not dependent on load order?

What would the desired default be, 'BIC' in all cases?

Thanks.

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
maintainer
#12 lore raw

Re: tcp congestion policy selection link order fragile

From: bert hubert <[email protected]>
Date: Mon, 18 Sep 2006 17:40:48 +0200

Quoted 1 line

What would the desired default be, 'BIC' in all cases?

And if BIC is not enabled in the configuration, then what?

#13 lore raw

Re: tcp congestion policy selection link order fragile

Quoted 3 lines · David Miller
Quoted 1 line

What would the desired default be, 'BIC' in all cases?

And if BIC is not enabled in the configuration, then what?

As the source notes "/* we'll always have reno */ ". This would make the
policy: the default is "bic" if available, otherwise it is "reno", which is *always* available.

But it is all up to you. I'm willing to do the leg work.

Bert

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
#14 lore raw

PATCH 1/1 tcp: set congestion default through Kconfig

Bert's attempt was noble
It showed your desire for the truth
A simple path exists

---
 net/ipv4/Kconfig           |   39 +++++++++++++++++++++++++++++++++++++--
 net/ipv4/sysctl_net_ipv4.c |    7 +++++++
 net/ipv4/tcp_cong.c        |    2 +-
 3 files changed, 45 insertions(+), 3 deletions(-)
net/ipv4/Kconfig +37 -2
index 90f9136..e922c3a 100644
@@ -573,12 +573,47 @@ config TCP_CONG_VENO
 	loss packets.
 	See http://www.ntu.edu.sg/home5/ZHOU0022/papers/CPFu03a.pdf
 
+choice
+	prompt "Default TCP congestion control"
+	default DEFAULT_BIC
+	help
+	  Select the TCP congestion control that will be used by default
+	  for all connections.
+
+	config DEFAULT_BIC
+		bool "Bic" if TCP_CONG_BIC=y
+
+	config DEFAULT_CUBIC
+		bool "Cubic" if TCP_CONG_CUBIC=y
+
+	config DEFAULT_HTCP
+		bool "Htcp" if TCP_CONG_HTCP=y
+
+	config DEFAULT_VEGAS
+		bool "Vegas" if TCP_CONG_VEGAS=y
+
+	config DEFAULT_WESTWOOD
+		bool "Westwood" if TCP_CONG_WESTWOOD=y
+
+	config DEFAULT_RENO
+		bool "Reno"
+
+endchoice
+
 endmenu
 
-config TCP_CONG_BIC
-	tristate
+config DEFAULT_BIC
 	depends on !TCP_CONG_ADVANCED
 	default y
 
+config DEFAULT_TCP_CONG
+	string
+	default "bic" if DEFAULT_BIC
+	default "cubic" if DEFAULT_CUBIC
+	default "htcp" if DEFAULT_HTCP
+	default "vegas" if DEFAULT_VEGAS
+	default "westwood" if DEFAULT_WESTWOOD
+	default "reno" if DEFAULT_RENO
+
 source "net/ipv4/ipvs/Kconfig"
 
net/ipv4/sysctl_net_ipv4.c +7 -0
index 19b2071..52b6481 100644
@@ -129,6 +129,13 @@ static int sysctl_tcp_congestion_control
 	return ret;
 }
 
+static __init void tcp_congestion_default(void)
+{
+	tcp_set_default_congestion_control(CONFIG_DEFAULT_TCP_CONG)
+}
+
+late_initcall(tcp_congestion_default);
+
 
 ctl_table ipv4_table[] = {
         {
net/ipv4/tcp_cong.c +1 -1
index 7ff2e42..af0aca1 100644
@@ -48,7 +48,7 @@ int tcp_register_congestion_control(stru
 		printk(KERN_NOTICE "TCP %s already registered\n", ca->name);
 		ret = -EEXIST;
 	} else {
-		list_add_rcu(&ca->list, &tcp_cong_list);
+		list_add_tail_rcu(&ca->list, &tcp_cong_list);
 		printk(KERN_INFO "TCP %s registered\n", ca->name);
 	}
 	spin_unlock(&tcp_cong_list_lock);
Signature · 1 line
1.4.1
#15 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig

Quoted 6 lines
---
 net/ipv4/Kconfig           |   39 +++++++++++++++++++++++++++++++++++++--
 net/ipv4/sysctl_net_ipv4.c |    7 +++++++
 net/ipv4/tcp_cong.c        |    2 +-
 3 files changed, 45 insertions(+), 3 deletions(-)

Nice solution.

Signature · 7 lines
Ian McDonald
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.com
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand
#16 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig

In reply to Stephen Hemminger
Quoted 3 lines · Stephen Hemminger

Bert's attempt was noble
It showed your desire for the truth
A simple path exists

I guess most people won't have a clue on what to configure here (especially with such sparse help text, but even with more it would be hard).

And you might end up with some weird error reports if someone manages to configure one of the more broken algorithms.

How about a single auto selection heuristic: e.g. check the handshake latencies and if they are too long switch from reno to the newer one deemed most stable?

-Andi

#17 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig

Quoted 3 lines · * Andi Kleen | 2006-09-19 12:03:51 [+0200]:

How about a single auto selection heuristic: e.g. check the handshake latencies and if they are too long switch from reno to the newer one deemed most stable?

Thats absolute no practicable solution to discover the 'right' algorithm. The latenzy is only one piece in the puzzle the determine the optimal congestion control algorithm and completely inadequate. You must also discover the bandwith and other factors to make a decision. And this is nearly impossible and object of actual research.

Through due to the heterogeneity of the net it is often not possible the select the right one. To select a "general purpose algorithm" here is the better choice.

BTW: BIC is sometimes to aggressive in comparison to standard tcp behaviour (e.g. in short RTT environments) - this is an known issue. Why not cubic as the default one?

Quoted 1 line

-Andi

HGN

Signature · 3 lines
"You need the computing power of a Pentium, 16 MB RAM and 1 GB Harddisk
to run Win95. It took the computing power of 3 Commodore 64
to fly to the Moon. Something is wrong here, and it wasn't the Apollo."
maintainer
#18 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig

From: Hagen Paul Pfeifer <[email protected]>
Date: Tue, 19 Sep 2006 14:35:35 +0200

Quoted 3 lines

BTW: BIC is sometimes to aggressive in comparison to standard tcp behaviour (e.g. in short RTT environments) - this is an known issue. Why not cubic as the default one?

We may change over to CUBIC as the default in 2.6.19, I wanted CUBIC to sit around in the tree for a while in case some major issues came up first.

#19 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig

Quoted 10 lines · On Tue, 19 Sep 2006 09:10:37 -0700 (PDT) David Miller <[email protected]>…

From: Hagen Paul Pfeifer <[email protected]>
Date: Tue, 19 Sep 2006 14:35:35 +0200

Quoted 3 lines

BTW: BIC is sometimes to aggressive in comparison to standard tcp behaviour (e.g. in short RTT environments) - this is an known issue. Why not cubic as the default one?

We may change over to CUBIC as the default in 2.6.19, I wanted CUBIC to sit around in the tree for a while in case some major issues came up first.

I was going to do this for 2.6.19 as well, but wanted more discussion and also do it as a separate step from the config changes.

Signature · 1 line
Stephen Hemminger <[email protected]>
#20 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig

In reply to Andi Kleen
Quoted 5 lines · On Tue, 19 Sep 2006 12:03:51 +0200 Andi Kleen <[email protected]> wrote:
Quoted 3 lines · Stephen Hemminger

Bert's attempt was noble
It showed your desire for the truth
A simple path exists

Quoted 7 lines

I guess most people won't have a clue on what to configure here (especially with such sparse help text, but even with more it would be hard).

And you might end up with some weird error reports if someone manages to configure one of the more broken algorithms.

I tried to only put known stable ones in the choices list.

maintainer
#22 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig

In reply to Stephen Hemminger

Actually, I take that back:

net/ipv4/sysctl_net_ipv4.c: In function $,1rx(Btcp_congestion_default$,1ry(B:
net/ipv4/sysctl_net_ipv4.c:134: error: syntax error before $,1rx(B}$,1ry(B token net/ipv4/sysctl_net_ipv4.c: At top level:
net/ipv4/sysctl_net_ipv4.c:136: warning: initialization from incompatible pointer type

I'll fix this up, but I'm seeing a very disturbing pattern lately of very clueful people posting patches without typing "make".

Is this too much to ask these days? This costs me an enormous amount of time, to review someone's patch and verify it's logical correctness, only to check it in and find out that it doesn't even build. That, frankly, is unacceptable.

maintainer
#23 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig

In reply to Stephen Hemminger

Ok, I really give up, after adding the missing semicolon I now get this:

net/ipv4/sysctl_net_ipv4.c:136: warning: initialization from incompatible pointer type

This is getting rediculious.

Folks, don't post patches which are not "request for comments" explicitly in the subject line without testing the build, ever!

#24 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig

It built (and ran) for me. But that was on a different machine. Perhaps there was some git confusion, I'll go back to using quilt it gives me less grief.

Signature · 1 line
Stephen Hemminger <[email protected]>
#25 lore raw

PATCH 1/1 tcp: set congestion default through Kconfig (v2)

In reply to David Miller

Bert's attempt was noble
It showed your desire for the truth
A simple path existed

---
 net/ipv4/Kconfig           |   39 +++++++++++++++++++++++++++++++++++++--
 net/ipv4/sysctl_net_ipv4.c |    6 ++++++
 net/ipv4/tcp_cong.c        |    2 +-
 3 files changed, 44 insertions(+), 3 deletions(-)
net/ipv4/Kconfig +37 -2
@@ -573,12 +573,47 @@
 	loss packets.
 	See http://www.ntu.edu.sg/home5/ZHOU0022/papers/CPFu03a.pdf
 
+choice
+	prompt "Default TCP congestion control"
+	default DEFAULT_BIC
+	help
+	  Select the TCP congestion control that will be used by default
+	  for all connections.
+
+	config DEFAULT_BIC
+		bool "Bic" if TCP_CONG_BIC=y
+
+	config DEFAULT_CUBIC
+		bool "Cubic" if TCP_CONG_CUBIC=y
+
+	config DEFAULT_HTCP
+		bool "Htcp" if TCP_CONG_HTCP=y
+
+	config DEFAULT_VEGAS
+		bool "Vegas" if TCP_CONG_VEGAS=y
+
+	config DEFAULT_WESTWOOD
+		bool "Westwood" if TCP_CONG_WESTWOOD=y
+
+	config DEFAULT_RENO
+		bool "Reno"
+
+endchoice
+
 endmenu
 
-config TCP_CONG_BIC
-	tristate
+config DEFAULT_BIC
 	depends on !TCP_CONG_ADVANCED
 	default y
 
+config DEFAULT_TCP_CONG
+	string
+	default "bic" if DEFAULT_BIC
+	default "cubic" if DEFAULT_CUBIC
+	default "htcp" if DEFAULT_HTCP
+	default "vegas" if DEFAULT_VEGAS
+	default "westwood" if DEFAULT_WESTWOOD
+	default "reno" if DEFAULT_RENO
+
 source "net/ipv4/ipvs/Kconfig"
 
net/ipv4/sysctl_net_ipv4.c +6 -0
@@ -129,6 +129,12 @@
 	return ret;
 }
 
+static int __init tcp_congestion_default(void)
+{
+	return tcp_set_default_congestion_control(CONFIG_DEFAULT_TCP_CONG);
+}
+
+late_initcall(tcp_congestion_default);
 
 ctl_table ipv4_table[] = {
         {
net/ipv4/tcp_cong.c +1 -1
@@ -48,7 +48,7 @@
 		printk(KERN_NOTICE "TCP %s already registered\n", ca->name);
 		ret = -EEXIST;
 	} else {
-		list_add_rcu(&ca->list, &tcp_cong_list);
+		list_add_tail_rcu(&ca->list, &tcp_cong_list);
 		printk(KERN_INFO "TCP %s registered\n", ca->name);
 	}
 	spin_unlock(&tcp_cong_list_lock);
maintainer Applied
#26 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig (v2)

From: Stephen Hemminger <[email protected]>
Date: Tue, 19 Sep 2006 13:41:32 -0700

Quoted 5 lines

Bert's attempt was noble
It showed your desire for the truth
A simple path existed

Applied, but...

net/ipv4/Kconfig:607:warning: defaults for choice values not supported

Hmmm...

#27 lore raw

PATCH 1/1 tcp: simpler bic default

New tcp default config
means we can eliminate the hack
that was used previous

net/ipv4/Kconfig +1 -4
@@ -602,10 +602,6 @@
 
 endmenu
 
-config DEFAULT_BIC
-	depends on !TCP_CONG_ADVANCED
-	default y
-
 config DEFAULT_TCP_CONG
 	string
 	default "bic" if DEFAULT_BIC
@@ -614,6 +610,7 @@
 	default "vegas" if DEFAULT_VEGAS
 	default "westwood" if DEFAULT_WESTWOOD
 	default "reno" if DEFAULT_RENO
+	default "bic"
 
 source "net/ipv4/ipvs/Kconfig"
 
#28 lore raw

Re: [PATCH] tcp: simpler bic default

Quoted 1 line · Stephen Hemminger

# CONFIG_TCP_CONG_ADVANCED is not set
# CONFIG_DEFAULT_BIC is not set
# CONFIG_DEFAULT_CUBIC is not set
# CONFIG_DEFAULT_HTCP is not set
# CONFIG_DEFAULT_VEGAS is not set
# CONFIG_DEFAULT_WESTWOOD is not set
# CONFIG_DEFAULT_RENO is not set
CONFIG_DEFAULT_TCP_CONG="bic"

There is no "bic" in the kernel now - will this do the right thing?

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
maintainer
#29 lore raw

Re: [PATCH] tcp: simpler bic default

From: bert hubert <[email protected]>
Date: Tue, 19 Sep 2006 23:43:05 +0200

Quoted 12 lines · Stephen Hemminger
Quoted 1 line

# CONFIG_TCP_CONG_ADVANCED is not set
# CONFIG_DEFAULT_BIC is not set
# CONFIG_DEFAULT_CUBIC is not set
# CONFIG_DEFAULT_HTCP is not set
# CONFIG_DEFAULT_VEGAS is not set
# CONFIG_DEFAULT_WESTWOOD is not set
# CONFIG_DEFAULT_RENO is not set
CONFIG_DEFAULT_TCP_CONG="bic"

Quoted 1 line

There is no "bic" in the kernel now - will this do the right thing?

This is wrong.

When TCP_CONG_ADVANCED is disabled, TCP_CONG_BIC should always be set and be the default, but that's not what the new code is doing.

I'm reverting Stephen's patch (again) until this is all sorted out as things are still broken in various ways even with the followon patch.

#30 lore raw

Re: [PATCH] tcp: simpler bic default

Quoted 7 lines

When TCP_CONG_ADVANCED is disabled, TCP_CONG_BIC should always be set and be the default, but that's not what the new code is doing.

I'm reverting Stephen's patch (again) until this is all sorted out as things are still broken in various ways even with the followon patch.

I don't see what's wrong, this is on a kernel with the two patches:

After "make defconfig"
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_DEFAULT_TCP_CONG="bic"
# CONFIG_NETFILTER_XT_MATCH_TCPMSS is not set
# CONFIG_IP_NF_TARGET_TCPMSS is not set
# CONFIG_ISCSI_TCP is not set

Enable TCP_ADVANCED_CONG with make menuconfig

CONFIG_TCP_CONG_ADVANCED=y
# TCP congestion control
CONFIG_TCP_CONG_BIC=y
CONFIG_TCP_CONG_CUBIC=m
CONFIG_TCP_CONG_WESTWOOD=m
CONFIG_TCP_CONG_HTCP=m
# CONFIG_TCP_CONG_HSTCP is not set
# CONFIG_TCP_CONG_HYBLA is not set
# CONFIG_TCP_CONG_VEGAS is not set
# CONFIG_TCP_CONG_SCALABLE is not set
# CONFIG_TCP_CONG_LP is not set
# CONFIG_TCP_CONG_VENO is not set
# CONFIG_DEFAULT_HTCP is not set
CONFIG_DEFAULT_TCP_CONG="bic"

After "make allmodconfig"
CONFIG_INET_TCP_DIAG=m
CONFIG_TCP_CONG_ADVANCED=y
# TCP congestion control
CONFIG_TCP_CONG_BIC=m
CONFIG_TCP_CONG_CUBIC=m
CONFIG_TCP_CONG_WESTWOOD=m
CONFIG_TCP_CONG_HTCP=m
CONFIG_TCP_CONG_HSTCP=m
CONFIG_TCP_CONG_HYBLA=m
CONFIG_TCP_CONG_VEGAS=m
CONFIG_TCP_CONG_SCALABLE=m
CONFIG_TCP_CONG_LP=m
CONFIG_TCP_CONG_VENO=m
# CONFIG_DEFAULT_HTCP is not set
CONFIG_DEFAULT_TCP_CONG="reno"

After "make allyesconfig"
CONFIG_INET_TCP_DIAG=y
CONFIG_TCP_CONG_ADVANCED=y
# TCP congestion control
CONFIG_TCP_CONG_BIC=y
CONFIG_TCP_CONG_CUBIC=y
CONFIG_TCP_CONG_WESTWOOD=y
CONFIG_TCP_CONG_HTCP=y
CONFIG_TCP_CONG_HSTCP=y
CONFIG_TCP_CONG_HYBLA=y
CONFIG_TCP_CONG_VEGAS=y
CONFIG_TCP_CONG_SCALABLE=y
CONFIG_TCP_CONG_LP=y
CONFIG_TCP_CONG_VENO=y
# CONFIG_DEFAULT_HTCP is not set
CONFIG_DEFAULT_TCP_CONG="bic"

maintainer
#31 lore raw

Re: [PATCH] tcp: simpler bic default

From: Stephen Hemminger <[email protected]>
Date: Tue, 19 Sep 2006 16:02:02 -0700

Quoted 9 lines
Quoted 7 lines

When TCP_CONG_ADVANCED is disabled, TCP_CONG_BIC should always be set and be the default, but that's not what the new code is doing.

I'm reverting Stephen's patch (again) until this is all sorted out as things are still broken in various ways even with the followon patch.

I don't see what's wrong, this is on a kernel with the two patches:

Look more closely:

Quoted 6 lines

After "make defconfig"
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_DEFAULT_TCP_CONG="bic"
# CONFIG_NETFILTER_XT_MATCH_TCPMSS is not set
# CONFIG_IP_NF_TARGET_TCPMSS is not set
# CONFIG_ISCSI_TCP is not set

But TCP_CONG_BIC isn't enabled so it won't get built nor be the default algorithm. When CONG_ADVANCED is off, as it is here, TCP_CONG_BIC should be "y". That's what happens before your changes.

#32 lore raw

Re: [PATCH] tcp: simpler bic default

Quoted 22 lines · On Tue, 19 Sep 2006 16:04:41 -0700 (PDT) David Miller <[email protected]>…

From: Stephen Hemminger <[email protected]>
Date: Tue, 19 Sep 2006 16:02:02 -0700

Quoted 9 lines
Quoted 7 lines

When TCP_CONG_ADVANCED is disabled, TCP_CONG_BIC should always be set and be the default, but that's not what the new code is doing.

I'm reverting Stephen's patch (again) until this is all sorted out as things are still broken in various ways even with the followon patch.

I don't see what's wrong, this is on a kernel with the two patches:

Look more closely:

Quoted 6 lines

After "make defconfig"
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_DEFAULT_TCP_CONG="bic"
# CONFIG_NETFILTER_XT_MATCH_TCPMSS is not set
# CONFIG_IP_NF_TARGET_TCPMSS is not set
# CONFIG_ISCSI_TCP is not set

Quoted 4 lines

But TCP_CONG_BIC isn't enabled so it won't get built nor be the default algorithm. When CONG_ADVANCED is off, as it is here, TCP_CONG_BIC should be "y". That's what happens before your changes.

Okay, build testing all the possibilities now, answer by morning..

Signature · 1 line
Stephen Hemminger <[email protected]>
#33 lore raw

Re: [PATCH] tcp: simpler bic default

Quoted 1 line · Stephen Hemminger

Okay, build testing all the possibilities now, answer by morning..

Please boot some of them as well - I can see a kernel that really wants to load "bic" at boot time but can't find it.

Bert

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
#34 lore raw

PATCH 1/1 tcp: default congestion control menu

Change how default TCP congestion control is chosen. Don't just use last installed module, instead allow selection during configuration, and make sure and use the default regardless of load order.

---
 net/ipv4/Kconfig           |   45 ++++++++++++++++++++++++++++++++++++++++-----
 net/ipv4/sysctl_net_ipv4.c |    6 ++++++
 net/ipv4/tcp_cong.c        |    2 +-
 3 files changed, 47 insertions(+), 6 deletions(-)
net/ipv4/Kconfig +40 -5
@@ -447,7 +447,7 @@
 	depends on INET_DIAG
 	def_tristate INET_DIAG
 
-config TCP_CONG_ADVANCED
+menuconfig TCP_CONG_ADVANCED
 	bool "TCP: advanced congestion control"
 	---help---
 	  Support for selection of various TCP congestion control
@@ -458,9 +458,7 @@
 
 	  If unsure, say N.
 
-# TCP Reno is builtin (required as fallback)
-menu "TCP congestion control"
-	depends on TCP_CONG_ADVANCED
+if TCP_CONG_ADVANCED
 
 config TCP_CONG_BIC
 	tristate "Binary Increase Congestion (BIC) control"
@@ -573,12 +571,49 @@
 	loss packets.
 	See http://www.ntu.edu.sg/home5/ZHOU0022/papers/CPFu03a.pdf
 
-endmenu
+choice
+	prompt "Default TCP congestion control"
+	default DEFAULT_BIC
+	help
+	  Select the TCP congestion control that will be used by default
+	  for all connections.
+
+	config DEFAULT_BIC
+		bool "Bic" if TCP_CONG_BIC=y
+
+	config DEFAULT_CUBIC
+		bool "Cubic" if TCP_CONG_CUBIC=y
+
+	config DEFAULT_HTCP
+		bool "Htcp" if TCP_CONG_HTCP=y
+
+	config DEFAULT_VEGAS
+		bool "Vegas" if TCP_CONG_VEGAS=y
+
+	config DEFAULT_WESTWOOD
+		bool "Westwood" if TCP_CONG_WESTWOOD=y
+
+	config DEFAULT_RENO
+		bool "Reno"
+
+endchoice
+
+endif
 
 config TCP_CONG_BIC
 	tristate
 	depends on !TCP_CONG_ADVANCED
 	default y
 
+config DEFAULT_TCP_CONG
+	string
+	default "bic" if DEFAULT_BIC
+	default "cubic" if DEFAULT_CUBIC
+	default "htcp" if DEFAULT_HTCP
+	default "vegas" if DEFAULT_VEGAS
+	default "westwood" if DEFAULT_WESTWOOD
+	default "reno" if DEFAULT_RENO
+	default "bic"
+
 source "net/ipv4/ipvs/Kconfig"
 
net/ipv4/sysctl_net_ipv4.c +6 -0
@@ -129,6 +129,12 @@
 	return ret;
 }
 
+static int __init tcp_congestion_default(void)
+{
+	return tcp_set_default_congestion_control(CONFIG_DEFAULT_TCP_CONG);
+}
+
+late_initcall(tcp_congestion_default);
 
 ctl_table ipv4_table[] = {
         {
net/ipv4/tcp_cong.c +1 -1
@@ -48,7 +48,7 @@
 		printk(KERN_NOTICE "TCP %s already registered\n", ca->name);
 		ret = -EEXIST;
 	} else {
-		list_add_rcu(&ca->list, &tcp_cong_list);
+		list_add_tail_rcu(&ca->list, &tcp_cong_list);
 		printk(KERN_INFO "TCP %s registered\n", ca->name);
 	}
 	spin_unlock(&tcp_cong_list_lock);
maintainer Applied
#35 lore raw

Re: [PATCH] tcp: default congestion control menu

From: Stephen Hemminger <[email protected]>
Date: Wed, 20 Sep 2006 13:28:06 -0700

Quoted 5 lines

Change how default TCP congestion control is chosen. Don't just use last installed module, instead allow selection during configuration, and make sure and use the default regardless of load order.

Applied, thanks Stephen.

#36 lore raw

PATCH 1/1 tcp: make cubic the default

In reply to bert hubert

Change default congestion control used from BIC to the newer CUBIC which it the successor to BIC but has better properties over long delay links.

---
 net/ipv4/Kconfig |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
net/ipv4/Kconfig +6 -6
@@ -454,7 +454,7 @@
 	  modules.
 
 	  Nearly all users can safely say no here, and a safe default
-	  selection will be made (BIC-TCP with new Reno as a fallback).
+	  selection will be made (CUBIC with new Reno as a fallback).
 
 	  If unsure, say N.
 
@@ -462,7 +462,7 @@
 
 config TCP_CONG_BIC
 	tristate "Binary Increase Congestion (BIC) control"
-	default y
+	default m
 	---help---
 	BIC-TCP is a sender-side only change that ensures a linear RTT
 	fairness under large windows while offering both scalability and
@@ -476,7 +476,7 @@
 
 config TCP_CONG_CUBIC
 	tristate "CUBIC TCP"
-	default m
+	default y
 	---help---
 	This is version 2.0 of BIC-TCP which uses a cubic growth function
 	among other techniques.
@@ -573,7 +573,7 @@
 
 choice
 	prompt "Default TCP congestion control"
-	default DEFAULT_BIC
+	default DEFAULT_CUBIC
 	help
 	  Select the TCP congestion control that will be used by default
 	  for all connections.
@@ -600,7 +600,7 @@
 
 endif
 
-config TCP_CONG_BIC
+config TCP_CONG_CUBIC
 	tristate
 	depends on !TCP_CONG_ADVANCED
 	default y
@@ -613,7 +613,7 @@
 	default "vegas" if DEFAULT_VEGAS
 	default "westwood" if DEFAULT_WESTWOOD
 	default "reno" if DEFAULT_RENO
-	default "bic"
+	default "cubic"
 
 source "net/ipv4/ipvs/Kconfig"
 
#37 lore raw

tested: Re: [PATCH] tcp: make cubic the default

Stephen,

I've applied both of your patches
(http://marc.theaimsgroup.com/?l=linux-netdev&m=115878447914612&w=2
and
http://marc.theaimsgroup.com/?l=linux-netdev&m=115878448125216&w=2
) and tried to break them, but it now appears to do the right thing in all cases, even when malforming the .config by hand, a 'make oldconfig' restores sanity.

Reno is chosen if none of the non-scary congestion avoidance algorithms are available, and the default for when they are are as you intended.

I've testbooted the resulting kernel and everything appears to work as desired, the proper TCP gets chosen, loading other ones does not change the default, but does make them available.

Unloading the module containing the configured policy sets the policy to 'cubic', which is probably the next entry in the policy list.

All in all, this final iteration of the congestion selection patches appears to do the job!

Davem, I'd recommend both patches for merging.

Bert

Quoted 73 lines · Stephen Hemminger

Change default congestion control used from BIC to the newer CUBIC which it the successor to BIC but has better properties over long delay links.

---
 net/ipv4/Kconfig |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
net/ipv4/Kconfig +6 -7
@@ -454,7 +454,7 @@
 	  modules.
 
 	  Nearly all users can safely say no here, and a safe default
-	  selection will be made (BIC-TCP with new Reno as a fallback).
+	  selection will be made (CUBIC with new Reno as a fallback).
 
 	  If unsure, say N.
 
@@ -462,7 +462,7 @@
 
 config TCP_CONG_BIC
 	tristate "Binary Increase Congestion (BIC) control"
-	default y
+	default m
 	---help---
 	BIC-TCP is a sender-side only change that ensures a linear RTT
 	fairness under large windows while offering both scalability and
@@ -476,7 +476,7 @@
 
 config TCP_CONG_CUBIC
 	tristate "CUBIC TCP"
-	default m
+	default y
 	---help---
 	This is version 2.0 of BIC-TCP which uses a cubic growth function
 	among other techniques.
@@ -573,7 +573,7 @@
 
 choice
 	prompt "Default TCP congestion control"
-	default DEFAULT_BIC
+	default DEFAULT_CUBIC
 	help
 	  Select the TCP congestion control that will be used by default
 	  for all connections.
@@ -600,7 +600,7 @@
 
 endif
 
-config TCP_CONG_BIC
+config TCP_CONG_CUBIC
 	tristate
 	depends on !TCP_CONG_ADVANCED
 	default y
@@ -613,7 +613,7 @@
 	default "vegas" if DEFAULT_VEGAS
 	default "westwood" if DEFAULT_WESTWOOD
 	default "reno" if DEFAULT_RENO
-	default "bic"
+	default "cubic"
 
 source "net/ipv4/ipvs/Kconfig"
 
-

To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

!DSPAM:4511a594269391527717022!

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services
maintainer
#38 lore raw

Re: tested: Re: [PATCH] tcp: make cubic the default

From: bert hubert <[email protected]>
Date: Sat, 23 Sep 2006 13:14:34 +0200

Quoted 4 lines

All in all, this final iteration of the congestion selection patches appears to do the job!

Davem, I'd recommend both patches for merging.

Great, I'll make sure I review them too and integrate them.

Thanks for checking this stuff out so thoroughly.

#40 lore raw

Re: [PATCH] tcp: set congestion default through Kconfig (v2)

In reply to David Miller
Quoted 2 lines · David Miller
Quoted 2 lines

Bert's attempt was noble
It showed your desire for the truth

It was also crap :-)

Quoted 2 lines

Applied, but...
net/ipv4/Kconfig:607:warning: defaults for choice values not supported

It does appear to do the right thing in all cases I throw against it, but this warning is sure to generate noise.

It probably means CONFIG_BIC is both available in the menu, and set as a separate option, and that this does not set the menu to CONFIG_BIC. This is not a big deal however as the menu is not shown at all whenever line 607 applies.

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services