tune back idle cwnd closing?

My apologies if this is a FAQ, I couldn't find it in the archives.

We have some dudes who are syncing large amounts of data across a dedicated long fat pipe at somewhat irregular intervals that are, sadly, longer than the rto. They feel the pain of having to reopen the window between transmissions.

Is there room for a compromise tunable that would be less aggressive about closing cwnd during idle periods but which wouldn't violate the spirit of 2861? No one wants broken TCP here.

They mention that Solaris has the tcp_slow_start_after_idle tunable and that it helps their situation. I mention that only as a data point, I wouldn't be foolish enough to try and use the presence of something in Solaris as justification :)

- z

Yours is the first complaint of this kind I recall seeing, but I've expected for a while someone would have this type of problem. RFC2861 seems conceptually nice at first, but there are a few things about it that bother me. One thing in particular is that a naturally bursty application (like yours) will actually perform better by padding its connection with junk data whenever it doesn't have real data to send. Or equivalently, it's punished for not sending data when it doesn't need to. I also think it may not do much good when there are connections with significantly different RTTs.

Given that RFC2681 is Experimental (and I'm not aware of any current efforts in the IETF to push it to the standard track), IHMO it would not be inappropriate to make this behavior controlled via sysctl.

Thanks,
-John

From: John Heffner <[email protected]>
Date: Tue, 25 Apr 2006 10:27:37 -0400

I have to respectfully disagree.

This is the price you pay when the network's congestion is being measured by probing, information becomes stale over time if you don't send any probes.

And this change of congestion state is real and happens frequently for most end to end users.

When you're bursty application is not sending, other flows can take up the pipe space you are not using, and you must reprobe to figure that out.

If the "restarted" connection does normal slow-start, one of two things will happen yes? Either it will grow its cwnd to >= the receiver's window, or it will have to stop before then because it triggered a packet loss.

In the first case, seems it would have been just as good to let the connection burst.

In the second case, is the effect on other connections really any better than if the connection just started-up from where it was before?

BTW, is the RFC 2681? I looked that one up on ietf.org and the RFC by that number was a different beast entirely - at least at a very quick glance.

rick jones

http://www.faqs.org/rfcs/rfc2861.html

Long periods when the sender is application-limited can lead to the invalidation of the congestion window. During periods when the TCP sender is network-limited, the value of the congestion window is repeatedly "revalidated" by the successful transmission of a window of data without loss. When the TCP sender is network-limited, there is an incoming stream of acknowledgements that "clocks out" new data, giving concrete evidence of recent available bandwidth in the network. In contrast, during periods when the TCP sender is application-limited, the estimate of available capacity represented by the congestion window may become steadily less accurate over time. In particular, capacity that had once been used by the network- limited connection might now be used by other traffic.

thanks.

May, might, could... :)

What concerned me the most was section 5, where the experiments were for dial-up connections and an interactive user then cat'ing a large file to the screen. How often does someone "list a moderately large file" without using less or more? And the bit about the second experiment with the real modem bank not showing any difference in what the user experienced because the bank had buffering was interesting. It suggests
(to me anyway) that perhaps the TCP receive window was too large for a
modem connection in the first place. Leaves me wondering what effect Linux's moderated receive window would have on that experiment.

rick jones

From: Rick Jones <[email protected]>
Date: Wed, 26 Apr 2006 15:16:18 -0700

Congestion window validation is the correct RFC.

OK, thanks for taking the time to look at it.

- z

A lot of the time doing 2861 is a good thing, since if you have a long pause, you've lost your ack clock, and you don't want to send a window-sized burst because you'll probably overflow a queue somewhere and step on your own feet. Since we don't have a pacing mechanism, a slow start is really the only way to do this.

I don't entirely buy the "staleness" argument. I don't think that *not* doing 2861 will affect the stability of congestion control, since all of the response mechanisms are still in place. (Most OS's don't do 2861, and it is not a standard.) If you have a long RTT, short RTT flows can make a big difference in congestion in a period much smaller than your timeout. In fact, congestion information is *always* stale by the time you get it. :)

Sometimes having cwnd validation turned on will make your applications perform better, sometimes worse. I don't think it would be incorrect to add a switch. One question is whether it's worth adding the switch (i.e., do enough people care?).

Myself, I'd be interested to see some quantitative comparisons of performance with a "real" application affected by this.

Thanks,
-John

From: John Heffner <[email protected]>
Date: Thu, 27 Apr 2006 13:47:33 -0400

Are you so sure? Doing cwnd timeout largely predates the congestion window validation work, in fact by several years.

In RFC 2581, it mentions Van Jacobson's recommendation of this idle period behavior, as just one example.

Your arguments about "all the feedback mechanisms are in place, so not reducing the cwnd after idle doesn't hurt congestion control" could be applied to the packet retransmit timeout handling of the congestion window, and I think that's kind of silly.

having looked now at both 2861 and the 99 paper it references I see lots of "may's" "mights" and "belief" but nothing "real world."

the CWV vs non CWV was done against a TCP that did indeed reset cwnd after an RTT of idle, so it wasn't showing reset at idle versus no reset at idle. just CWV's less draconian (?) reset than the non CWV stack.

the experimental validation in the 99 paper was still a simulation using dummynet and a number of buffers rather smaller than what modem banks were offering at the time, and it was for a modem, rather than any other sort of link. and when they did use a real modem, the buffering in the modem bank seems to have made the whole thing moot.

there was nothing about effect on intranets, or high-speed long hauls or any of that.

what that means wrt having a sysctl to enable/disable functionality still listed as experimental i'm not sure

rick jones