is non-inheritance of congestion control algorithm from the listen socket a bug or a feature?

Dave Taht asked me if I would consider adding control/reporting of the congestion control algorithm to netperf. I'm in the midst of doing that and have noticed that setting a valid congestion control algorithm on a listen endpoint does not seem to be inherited by subsequent connections.
Rather than rely on netperf/netserver I cobbled-together a quick test
program to show what I mean.

raj@raj-ubuntu-guest:~$ uname -a
Linux raj-ubuntu-guest 3.2.0-rc2+ #1 SMP Mon Nov 28 16:39:21 PST 2011 x86_64 x86_64 x86_64 GNU/Linux

raj@raj-ubuntu-guest:~$ find /proc -name "*cong*" -print -exec cat {} \; 2>/dev/null
/proc/sys/net/ipv4/tcp_congestion_control
cubic
/proc/sys/net/ipv4/tcp_available_congestion_control
cubic reno
/proc/sys/net/ipv4/tcp_allowed_congestion_control
cubic reno

# first just report what one gets
raj@raj-ubuntu-guest:~$ ./tcp_congestion_test
Will make no request to set congestion control algorithm
active socket has '' for congestion control
passive socket has '' for congestion control
after listen() passive socket has '' for congestion control
after connect() active has 'cubic' for congestion control
after accept() accepted has 'cubic' for congestion control

# now specify a valid alternative
raj@raj-ubuntu-guest:~$ ./tcp_congestion_test  reno
Will be requesting 'reno' as the congestion control algorithm
active socket has 'reno' for congestion control
passive socket has 'reno' for congestion control
after listen() passive socket has 'reno' for congestion control
after connect() active has 'reno' for congestion control
after accept() accepted has 'cubic' for congestion control

# now specify an invalid alternative just for completeness
raj@raj-ubuntu-guest:~$ ./tcp_congestion_test  bogus
Will be requesting 'bogus' as the congestion control algorithm
active socket has '' for congestion control
passive socket has '' for congestion control
after listen() passive socket has '' for congestion control
after connect() active has 'cubic' for congestion control
after accept() accepted has 'cubic' for congestion control

Is the non-inheritance of the congestion control algorithm a bug or a feature?

rick jones

Hey, thx for trying it!

Hmm... It's kind of both.

Recently I submitted an RFC patch to the
rsync mailing list, and it seemed nice to be able to set it on a per socket basis based on various configuration options in the rsync conf file and what was explicitly allowed.

That patch is here:

https://lists.samba.org/archive/rsync/2011-November/027111.html

It was also way cool to see tcp-lp in action vs other sorts of tcp.

That said we have a range of allowable congestion algorithms ranging from more intense than cubic to low bandwidth to wireless, to highly specialized (data center tcp), to some that require both sides to be running the same alg, others that are server only...

Your question sort of involves what should a user be able to set.

On the one hand, in the general case an app should not be allowed to choose a more aggressive tcp.

On the other hand, an app on a wireless box, requesting/getting, say, westwood over a default of lp or cubic sort of makes sense.

On the gripping hand I thiink pushing this level of decison making out to the app is just fine by me, so inherit or not, is the app's call to setsockopt...

And in netperf's case, that would be all the algs made available via allowable_congestion_control.

I had another idea regarding pushing alternate tcp algos out to more apps without requiring source code changes, which would be to create a cgroup controller, which would be both trendy and would provide 'inherit-ability'...

But first up I think would be for more folk to see how cool different algos can be.

Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net

IMHO its a bug.

Can you provide a fix ?

Thanks !

I can try once I find my way through the maze.

rick

I actually think it's a feature :)

I find it awkward to set CC on listening socket. And current document defines the sysctl well

tcp_congestion_control - STRING
Set the congestion control algorithm to be used for new connections. The algorithm "reno" is always available, but additional choices may be available based on kernel configuration. Default is set as part of kernel configuration.

This might be a feature, but contradicts most socket options set on listener and inherited by a child socket on accept()

tcp_congestion_control should be system wide default.

Anyway, "man 7 tcp" doesnt document TCP_CONGESTION, so we are not supposed to play with it ;)

Oh well...

From: Eric Dumazet <[email protected]>
Date: Tue, 29 Nov 2011 22:46:15 +0100

There is really no reason to keep the current behavior.

If an application sets the congestion control algorithm on a listening socket to a non-default value, what effect could possibly be intended?

Congestion control doesn't even come into play at all on a listening socket, therefore the only logical expectation is that it inherits to the child.

The only other logical behavior would be to forbid this operation on a listening socket, since it has no effect, but that doesn't make any sense now does it? :-)

Moreover, an application can use setsockopt(TCP_CONGESTION) before calling listen() (while socket is still in CLOSE state)

Agreed, it was just an oversight of the initial design. The setsockopt() on the listening socket is ignored.