TCP Illinois preliminary version

Preliminary version if TCP Illinois for 2.6.22. It needs more testing and refinement but it captures the basic concepts.

---
 net/ipv4/Kconfig        |   13 +++
 net/ipv4/Makefile       |    1 +
 net/ipv4/tcp_illinois.c |  238 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 252 insertions(+), 0 deletions(-)
 create mode 100644 net/ipv4/tcp_illinois.c

index dc61e66..e62aee0 100644
@@ -588,6 +588,19 @@ config TCP_CONG_YEAH
 	For further details look here:
 	  http://wil.cs.caltech.edu/pfldnet2007/paper/YeAH_TCP.pdf
 
+config TCP_CONG_ILLINOIS
+	tristate "TCP Illinois"
+	depends on EXPERIMENTAL
+	default n
+	---help---
+	TCP-Illinois is a sender-side modificatio of TCP Reno for
+	high speed long delay links. It uses round-trip-time to
+	adjust the alpha and beta parameters to achieve a higher average
+	throughput and maintain fairness.
+
+	For further details see:
+	  http://www.ews.uiuc.edu/~shaoliu/tcpillinois/index.html
+
 choice
 	prompt "Default TCP congestion control"
 	default DEFAULT_CUBIC

index eeb94d5..4ff6c15 100644
@@ -50,6 +50,7 @@ obj-$(CONFIG_TCP_CONG_VENO) += tcp_veno.o
 obj-$(CONFIG_TCP_CONG_SCALABLE) += tcp_scalable.o
 obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o
 obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o
+obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o
 obj-$(CONFIG_NETLABEL) += cipso_ipv4.o
 
 obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \

1.5.0.5

From: Stephen Hemminger <[email protected]>
Date: Mon, 2 Apr 2007 20:15:25 -0700

No objections from me and I don't see any obvious problems.

Would you like me to stuff this into net-2.6.22 now so that it can get some more exposure and people banging on it or would you like me to wait a bit?

Thanks.

I would like one of the originators to check the math.

In case you are worried, I received assurance from Shao Liu that they decided not to pursue patent and doing a Linux version was okay.

HI Stephen,

Do you wish me to check the modified code of the implementation in your first email yesterday? I have noticed the modification and have not had a chance to check that. I can check it tonight.

BTW, are all the changes in that first email?

Best regards!

Yours sincerely,
Shao Liu (Julian)

University of Illinois at Urbana Champaign
Coordinated Science Laboratory and Department of Electrical and Computer Engineering
www.ews.uiuc.edu/~shaoliu

-----Original Message-----
From: Stephen Hemminger [mailto:[email protected]]
Sent: Monday, April 02, 2007 10:45 PM
To: David Miller
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC] TCP Illinois preliminary version

I would like one of the originators to check the math.

In case you are worried, I received assurance from Shao Liu that they decided not to pursue patent and doing a Linux version was okay.

Hi, Stephen, David

Yes, you can go ahead to implement it in Linux. We are glad to see that TCP-Illinois gets more exposure by this way.

Best regards!

Yours sincerely,
Shao Liu (Julian)

University of Illinois at Urbana Champaign
Coordinated Science Laboratory and Department of Electrical and Computer Engineering
www.ews.uiuc.edu/~shaoliu

-----Original Message-----
From: Stephen Hemminger [mailto:[email protected]]
Sent: Monday, April 02, 2007 10:45 PM
To: David Miller
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC] TCP Illinois preliminary version

I would like one of the originators to check the math.

In case you are worried, I received assurance from Shao Liu that they decided not to pursue patent and doing a Linux version was okay.

This is a new implementation of TCP Illinois invented by Shao Liu at University of Illinois. It is a another variant of Reno which adapts the alpha and beta parameters based on RTT. The basic idea is to increase window less rapidly as delay approaches the maximum. See the papers and talks to get a more complete description.

Please consider for 2.6.22.

---
 net/ipv4/Kconfig        |   13 +++
 net/ipv4/Makefile       |    1 +
 net/ipv4/tcp_illinois.c |  212 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 226 insertions(+), 0 deletions(-)
 create mode 100644 net/ipv4/tcp_illinois.c

index dc61e66..e62aee0 100644
@@ -588,6 +588,19 @@ config TCP_CONG_YEAH
 	For further details look here:
 	  http://wil.cs.caltech.edu/pfldnet2007/paper/YeAH_TCP.pdf
 
+config TCP_CONG_ILLINOIS
+	tristate "TCP Illinois"
+	depends on EXPERIMENTAL
+	default n
+	---help---
+	TCP-Illinois is a sender-side modificatio of TCP Reno for
+	high speed long delay links. It uses round-trip-time to
+	adjust the alpha and beta parameters to achieve a higher average
+	throughput and maintain fairness.
+
+	For further details see:
+	  http://www.ews.uiuc.edu/~shaoliu/tcpillinois/index.html
+
 choice
 	prompt "Default TCP congestion control"
 	default DEFAULT_CUBIC

index eeb94d5..4ff6c15 100644
@@ -50,6 +50,7 @@ obj-$(CONFIG_TCP_CONG_VENO) += tcp_veno.o
 obj-$(CONFIG_TCP_CONG_SCALABLE) += tcp_scalable.o
 obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o
 obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o
+obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o
 obj-$(CONFIG_NETLABEL) += cipso_ipv4.o
 
 obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \

1.5.0.5

Hi Stephen,

Thanks for the implementation. I have checked the codes, and I have the following comments:

1, The math is basically correct in computing alpha and beta from da.

2, In the computation of da, you implicitly set d1=0. Setting d1=0 is a conservative approach, although it also works well (it may lead to a longer convergence time than a positive d1). The reason is that, even if far from congestion, there are some burstiness in RTT measurements, and we get da>d_min. This way, we have an alpha<alpha_max even if congestion is far away. We choose d1 to be close to 0 in our simulations for conservativeness, but we believe even if d1 is non-trivial positive, like 0.05*dm, it should work, and it might work better.

3, There is one more feature in TCP-Illinois: to avoid setting alpha to be alpha_max if actually the congestion level is high and the small queueing delay is due to noise. We have explained that in our updated version of the paper, available at
http://www.ews.uiuc.edu/~shaoliu/papersandslides/liubassri06perf.pdf
We have also implemented this feature in our ns-2 code. The basic idea of this feature is that, once da exceeds d1 and alpha<alpha_max, we know that congestion level is not very low. Once this happen, the congestion level should not drop unless some user has experienced a packet loss and backed off its window size. If that is the case, da should drop to below d1 for a long time. If, however, da drops to below d1 suddenly, but increases back to above d1 soon, we predict that congestion is not alleviated, and the small da measurement comes from noise. Under this case, we should not set alpha=alpha_max, but should set it to be its previous legitimate value.

We believe that this feature is important if there is noise in delay measurement and if d1 is set to be nonzero. Even if d1 is set to be zero, we think this feature should be included if the noise is delay measurement is significant. Under this case, we can modify the feature a little bit: alpha is initially computed to be around alpha_max when congestion is far. As da increases, alpha decreases. But alpha cannot be increased as soon as da drops. Alpha can only be increased if the decrease trend of da last for a certain period.

4, In your implementation, you set alpha_max=10 and alpha_min=0.1. Although this parameter setting is chosen by our paper, we suggest you try some different values, especially for alpha_min. An alpha_min from 0.2 to 0.5 might be a better choice, as this will make the convergence time faster.

Please feel free to tell me if you have any related questions.

Thanks!

Best regards!

Yours sincerely,
Shao Liu (Julian)

University of Illinois at Urbana Champaign
Coordinated Science Laboratory and Department of Electrical and Computer Engineering
www.ews.uiuc.edu/~shaoliu

-----Original Message-----
From: Stephen Hemminger [mailto:[email protected]]
Sent: Tuesday, April 03, 2007 11:17 AM
To: David Miller
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH] TCP Illinois congestion control

This is a new implementation of TCP Illinois invented by Shao Liu at University of Illinois. It is a another variant of Reno which adapts the alpha and beta parameters based on RTT. The basic idea is to increase window less rapidly as delay approaches the maximum. See the papers and talks to get a more complete description.

Please consider for 2.6.22.

---
 net/ipv4/Kconfig        |   13 +++
 net/ipv4/Makefile       |    1 +
 net/ipv4/tcp_illinois.c |  212
+++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 226 insertions(+), 0 deletions(-)  create mode 100644

net/ipv4/tcp_illinois.c

100644

@@ -588,6 +588,19 @@ config TCP_CONG_YEAH
 	For further details look here:
 	  http://wil.cs.caltech.edu/pfldnet2007/paper/YeAH_TCP.pdf
 
+config TCP_CONG_ILLINOIS
+	tristate "TCP Illinois"
+	depends on EXPERIMENTAL
+	default n
+	---help---
+	TCP-Illinois is a sender-side modificatio of TCP Reno for
+	high speed long delay links. It uses round-trip-time to
+	adjust the alpha and beta parameters to achieve a higher average
+	throughput and maintain fairness.
+
+	For further details see:
+	  http://www.ews.uiuc.edu/~shaoliu/tcpillinois/index.html
+
 choice
 	prompt "Default TCP congestion control"
 	default DEFAULT_CUBIC

100644

@@ -50,6 +50,7 @@ obj-$(CONFIG_TCP_CONG_VENO) += tcp_veno.o
 obj-$(CONFIG_TCP_CONG_SCALABLE) += tcp_scalable.o
 obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o
 obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o
+obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o
 obj-$(CONFIG_NETLABEL) += cipso_ipv4.o
 
 obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \ diff

--git a/net/ipv4/tcp_illinois.c b/net/ipv4/tcp_illinois.c new file mode 100644 index 0000000..f7c0b76

tp->snd_cwnd) {
+ if (tp->snd_cwnd < tp->snd_cwnd_clamp)
+ tp->snd_cwnd++;
+ tp->snd_cwnd_cnt = 0;
+ } else
+ tp->snd_cwnd_cnt++;
+ }
+}
+
+/*
+ * Beta used for multiplicative decrease.
+ * For small window sizes returns same value as Reno (0.5)
+ *
+ * If delay is small (10% of max) then beta = 1/8
+ * If delay is up to 80% of max then beta = 1/2
+ * In between is a linear function
+ */
+static inline u32 beta(const struct sock *sk) {
+ const struct tcp_sock *tp = tcp_sk(sk);
+ const struct tcp_illinois *ca = inet_csk_ca(sk);
+ u32 dm, d2, d3, da;
+
+ if (tp->snd_cwnd < win_thresh)
+ return BETA_BASE;
+
+ dm = ca->max_rtt - ca->min_rtt;
+ da = ca->avg_rtt - ca->min_rtt;
+
+ d2 = dm / 10;
+ if (da <= d2)
+ return BETA_MIN;
+ d3 = (8 * dm) / 10;
+ if (da >= d3 || d3 <= d2)
+ return BETA_MAX;
+
+ return (BETA_MIN * d3 - BETA_MAX * d2 + (BETA_MAX - BETA_MIN) * da)
+ / (d3 - d2);
+}
+
+static u32 tcp_illinois_ssthresh(struct sock *sk) {
+ struct tcp_sock *tp = tcp_sk(sk);
+
+ /* Multiplicative decrease */
+ return max((tp->snd_cwnd * beta(sk)) >> BETA_SHIFT, 2U); }
+
+/* Extract info for TCP socket info provided via netlink.
+ * We aren't really doing Vegas, but we can provide RTT info */ static +void tcp_illinois_get_info(struct sock *sk, u32 ext,
+ struct sk_buff *skb)
+{
+ const struct tcp_illinois *ca = inet_csk_ca(sk);
+
+ if (ext & (1 << (INET_DIAG_VEGASINFO - 1))) {
+ struct tcpvegas_info info = {
+ .tcpv_enabled = 1,
+ .tcpv_rtt = ca->avg_rtt,
+ .tcpv_minrtt = ca->min_rtt,
+ };
+
+ nla_put(skb, INET_DIAG_VEGASINFO, sizeof(info), &info);
+ }
+}
+
+static struct tcp_congestion_ops tcp_illinois = {
+ .init = tcp_illinois_init,
+ .ssthresh = tcp_illinois_ssthresh,
+ .min_cwnd = tcp_reno_min_cwnd,
+ .cong_avoid = tcp_illinois_cong_avoid,
+ .rtt_sample = tcp_illinois_rtt_calc,
+ .get_info = tcp_illinois_get_info,
+
+ .owner = THIS_MODULE,
+ .name = "illinois",
+};
+
+static int __init tcp_illinois_register(void) {
+ return tcp_register_congestion_control(&tcp_illinois);
+}
+
+static void __exit tcp_illinois_unregister(void) {
+ tcp_unregister_congestion_control(&tcp_illinois);
+}
+
+module_init(tcp_illinois_register); +module_exit(tcp_illinois_unregister);
+
+MODULE_AUTHOR("Stephen Hemminger, Shao Liu"); MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Illinois TCP");
--
1.5.0.5

This is an implementation of TCP Illinois invented by Shao Liu at University of Illinois. It is a another variant of Reno which adapts the alpha and beta parameters based on RTT. The basic idea is to increase window less rapidly as delay approaches the maximum. See the papers and talks to get a more complete description.

---
 net/ipv4/Kconfig        |   13 ++
 net/ipv4/Makefile       |    1 +
 net/ipv4/tcp_illinois.c |  284 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 298 insertions(+), 0 deletions(-)
 create mode 100644 net/ipv4/tcp_illinois.c

index dc61e66..e62aee0 100644
@@ -588,6 +588,19 @@ config TCP_CONG_YEAH
 	For further details look here:
 	  http://wil.cs.caltech.edu/pfldnet2007/paper/YeAH_TCP.pdf
 
+config TCP_CONG_ILLINOIS
+	tristate "TCP Illinois"
+	depends on EXPERIMENTAL
+	default n
+	---help---
+	TCP-Illinois is a sender-side modificatio of TCP Reno for
+	high speed long delay links. It uses round-trip-time to
+	adjust the alpha and beta parameters to achieve a higher average
+	throughput and maintain fairness.
+
+	For further details see:
+	  http://www.ews.uiuc.edu/~shaoliu/tcpillinois/index.html
+
 choice
 	prompt "Default TCP congestion control"
 	default DEFAULT_CUBIC

index eeb94d5..4ff6c15 100644
@@ -50,6 +50,7 @@ obj-$(CONFIG_TCP_CONG_VENO) += tcp_veno.o
 obj-$(CONFIG_TCP_CONG_SCALABLE) += tcp_scalable.o
 obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o
 obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o
+obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o
 obj-$(CONFIG_NETLABEL) += cipso_ipv4.o
 
 obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \

1.5.0.6

Hello,

We are currently using both 1 Gb & 10 Gb links, that interconnect several servers that are very *local* to each other. Typical RTT times range from 0.2 ms - 0.3 ms.

We are currently using TCP reno - is there a more suitable congestion control algorithm for our application, especially using the 10 Gb links ?
(Most of the High-Speed TCP algorithms seem suitable for large RTT, long-distance networks).

Thanks
David

________________________________________________________________________
AOL now offers free email to everyone. Find out more about what's free from AOL at AOL.com.

I'm not aware of any tests for high speed links with very low RTTs, but I suspect that the new algorithms will not change much, if the connections you have are indeed local than the Ethernet pause mechanism is more effective for the flow control you need.

Baruch

Thanks - so you are suggesting we enable 802.3 flow-control / pause-frames? (it's currently disabled)

-----Original Message-----
From: [email protected]
To: [email protected]
Cc: [email protected]; [email protected]
Sent: Wed, 4 Apr 2007 8:39 AM
Subject: Re: TCP congestion control for fast, short-distance networks ?

several
servers that are very *local* to

control
algorithm for our

long-distance networks).

I'm not aware of any tests for high speed links with very low RTTs, but I suspect that the new algorithms will not change much, if the connections you have are indeed local than the Ethernet pause mechanism is more effective for the flow control you need.

Baruch
-
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

________________________________________________________________________
AOL now offers free email to everyone. Find out more about what's free from AOL at AOL.com.

I do, but do test it before you bet on it. I've never tested such a scenario but from my experience the lower the rtt the lesser are the problems that the high speed algorithms are trying to solve.

Baruch

Pause helps (if the hardware works). But on very high speed links with low RTT, your main source of throttling is lack of bus bandwidth on receiver.

Stephen Hemminger <[email protected]>

From: Baruch Even <[email protected]>
Date: Wed, 4 Apr 2007 19:11:21 +0300

Congestion control, at the TCP level, is from one perspective completely senseless on local high-speed networks. The reason is that once any kind of congestion at that level is detected, the condition is gone by the time you could even possibly respond to it.

So with that in mind, reno is as good as anything and yes do use pause frames.

thanks for your help - I'll investigate turning on 802.3 level flow control

-----Original Message-----
From: [email protected]
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]
Sent: Wed, 4 Apr 2007 12:55 PM
Subject: Re: TCP congestion control for fast, short-distance networks ?

From: Baruch Even <[email protected]>
Date: Wed, 4 Apr 2007 19:11:21 +0300

pause-frames?

Congestion control, at the TCP level, is from one perspective completely senseless on local high-speed networks. The reason is that once any kind of congestion at that level is detected, the condition is gone by the time you could even possibly respond to it.

So with that in mind, reno is as good as anything and yes do use pause frames.
-
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

________________________________________________________________________
AOL now offers free email to everyone. Find out more about what's free from AOL at AOL.com.

From: Stephen Hemminger <[email protected]>
Date: Wed, 4 Apr 2007 08:07:43 -0700

I've applied this to net-2.6.22, let me know if there are any follow-on bug fixes I need to add as well.

Thanks!

I have a new version ready.