TCP-Hybla proposal
Subsystems (4)
PATCH 1/1 TCP-Hybla proposal
Hi
This is the official patch to implement TCP Hybla congestion avoidance.
- "In heterogeneous networks, TCP connections that incorporate a terrestrial or satellite radio link are greatly disadvantaged with respect to entirely wired connections, because of their longer round trip times (RTTs). To cope with this problem, a new TCP proposal, the TCP Hybla, is presented and discussed in the paper[1]. It stems from an analytical evaluation of the congestion window dynamics in the TCP standard versions (Tahoe, Reno, NewReno), which suggests the necessary modifications to remove the performance dependence on RTT.[...]"[1]
[1]: Carlo Caini, Rosario Firrincieli, "TCP Hybla: a TCP enhancement for heterogeneous networks",
International Journal of Satellite Communications and Networking Volume 22, Issue 5 , Pages 547 - 566. September 2004.
Please note that this patch:
- cleanly compiled against 2.6.11-rc4;
- reached very good results in terms of fairness and friendliness with other TCP standards and proposals during tests;
- gives a throughput very close to maximum fair share when satellite connections fight for bandwidth in a terrestrial bottleneck, and improves their performance also in case of no congestion;
- acts exactly as newreno when minimum rtt estimated by the sender is lower or equal to rtt0 (default=25 ms).
The protocol is obviously disabled by default and can be turned on by switching its sysctl, as usual.
As Mr.Yee-Ting Li pointed out in his last thread, the deployments of these experimental protocols are too premature and they should be switched OFF by default to prevent undesirable consequences to network stability.
One last note: IMHO we really need a better way to select congestion avoidance scheme between those available, instead of switching each one on and off. I.e., we can't say how vegas and westwood perform when switched on together, can we?
I'd really appreciate if you consider my work for apply. TIA,
Signature · 2 lines
Daniele Lacamera root at danielinux.net
Documentation/networking/ip-sysctl.txt +12 -0
@@ -349,6 +349,18 @@ window. Allows two flows sharing the same connection to converge more rapidly. Default: 1 + +tcp_hybla - BOOLEAN + Enable TCP-Hybla congestion control algorithm. + TCP-Hybla is a sender-side only change that eliminates penalization of + long-RTT, large-bandwidth connections, like when satellite legs are + involved, expecially when sharing a common bottleneck with normal + terrestrial connections. + Default: 0 + +tcp_hybla_rtt0 - INTEGER + Divisor to set up rtt0 value for hybla congestion control. + Default: 40 (= 1/40 sec == 25ms) tcp_default_win_scale - INTEGER Sets the minimum window scale TCP will negotiate for on all
include/linux/sysctl.h +2 -0
@@ -344,6 +344,8 @@NET_TCP_DEFAULT_WIN_SCALE =105 ,NET_TCP_MODERATE_RCVBUF =106 ,NET_TCP_TSO_WIN_DIVISOR =107 , +NET_TCP_HYBLA =108 , + NET_TCP_HYBLA_RTT0=109, }; enum {
include/linux/tcp.h +10 -0
@@ -434,6 +434,16 @@__u32 last_cwnd;/* the last snd_cwnd */ __u32 last_stamp;/* time when updated last_cwnd */ } bictcp; + +/* Tcp Hybla structure. */ +struct { +__u32 snd_cwnd_cents;/* Keeps increment values when it is <1, <<7 */ +__u32 rho;/* Rho parameter, integer part */ +__u32 rho2;/* Rho * Rho, integer part */ +__u32 rho_3ls;/* Rho parameter, <<3 */ +__u32 rho2_7ls;/* Rho^2, <<7 */ +__u32 minrtt;/* Minimum smoothed round trip time value seen */ + } hybla; };static inline struct tcp_sock *tcp_sk (const struct sock *sk)
include/net/tcp.h +39 -0
@@ -608,6 +608,8 @@extern int sysctl_tcp_bic_low_window;extern int sysctl_tcp_moderate_rcvbuf;extern int sysctl_tcp_tso_win_divisor; +extern int sysctl_tcp_hybla; +extern int sysctl_tcp_hybla_rtt0;extern atomic_t tcp_memory_allocated;extern atomic_t tcp_sockets_allocated;
@@ -2017,4 +2019,41 @@ return (cwnd != 0); } + +/* +* TCP HYBLA Functions and constants +*/ +/* Hybla reference round trip time (default= 1/40 sec = 25 ms), expressed in jiffies */ +#define RTT0 (__u32) ((HZ/sysctl_tcp_hybla_rtt0)) +/* +* This is called in tcp_ipv4.c and +* tcp_minisocks.c when connection starts +*/ +static inline void init_hybla (struct tcp_sock *tp) +{ + tp->hybla.rho =0 ; + tp->hybla.rho2 =0 ; + tp->hybla.rho_3ls =0 ; + tp->hybla.rho2_7ls =0 ; + tp->hybla.snd_cwnd_cents =0 ; + tp->hybla.avgrtt =0 ; +} +/* This is called to refresh values for hybla parameters */ +static inline void hybla_recalc_param (struct tcp_sock *tp) +{ + + tp->hybla.rho_3ls = (tp->srtt /RTT0 ); +if (tp->hybla.rho_3ls <8 ) + tp->hybla.rho_3ls =8 ; + + tp->hybla.rho=(tp->hybla.rho_3ls >>3 ); + tp->hybla.rho2_7ls = ((tp->hybla.rho_3ls * tp->hybla.rho_3ls)<<1 ); + tp->hybla.rho2=(tp->hybla.rho2_7ls >>7 ); + +if (sysctl_tcp_initial_bw_est!=0 ) +initial_bw_est_recalc_clamp (tp); + +if (sysctl_tcp_cong_avoid_style ==2 ) + tp->snd_cwnd_clamp =min_t (__u32, tp->snd_cwnd_clamp, tp->hybla.rho<<16 ); +}#endif /* _TCP_H */
net/ipv4/sysctl_net_ipv4.c +16 -0
@@ -682,6 +682,22 @@ .mode = 0644, .proc_handler = &proc_dointvec, }, + { + .ctl_name = NET_TCP_HYBLA, + .procname = "tcp_hybla", + .data = &sysctl_tcp_hybla, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &proc_dointvec + }, + { + .ctl_name = NET_TCP_HYBLA_RTT0, + .procname = "tcp_hybla_rtt0", + .data = &sysctl_tcp_hybla_rtt0, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &proc_dointvec + }, { .ctl_name = 0 } };
net/ipv4/tcp.c +4 -0
@@ -1813,6 +1813,10 @@if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK ))inet_reset_saddr (sk); + +/* [TCP HYBLA] Reset values on disconnect */ +if (sysctl_tcp_cong_avoid_style ==2 ) +init_hybla (tp); sk->sk_shutdown =0 ;sock_reset_flag (sk,SOCK_DONE );
net/ipv4/tcp_input.c +177 -3
219 lines not on this page · raw message
net/ipv4/tcp_ipv4.c +3 -0
@@ -2055,6 +2055,9 @@* efficiently to them. -DaveM */ tp->snd_cwnd =2 ; + +/* Reset hybla parameters on socket initialization. */ +init_hybla (tp);/* See draft-stevens-tcpca-spec-01 for discussion of the * initialization of these values.
net/ipv4/tcp_minisocks.c +3 -0
@@ -784,6 +784,9 @@ newtp->dsack =0 ; newtp->eff_sacks =0 ; + +/* Reset hybla parameters on socket initialization. */ +init_hybla (newtp); newtp->probes_out =0 ; newtp->num_sacks =0 ;
Quoted 2 lines · Daniele Lacamera
Hi
This is the official patch to implement TCP Hybla congestion
avoidance.
I've post a wrong/unclean patch. Here's the right one. Sorry.
Signature · 2 lines
Daniele Lacamera root at danielinux.net
Documentation/networking/ip-sysctl.txt +12 -0
@@ -349,6 +349,18 @@ window. Allows two flows sharing the same connection to converge more rapidly. Default: 1 + +tcp_hybla - BOOLEAN + Enable TCP-Hybla congestion control algorithm. + TCP-Hybla is a sender-side only change that eliminates penalization of + long-RTT, large-bandwidth connections, like when satellite legs are + involved, expecially when sharing a common bottleneck with normal + terrestrial connections. + Default: 0 + +tcp_hybla_rtt0 - INTEGER + Divisor to set up rtt0 value for hybla congestion control. + Default: 40 (= 1/40 sec == 25ms) tcp_default_win_scale - INTEGER Sets the minimum window scale TCP will negotiate for on all
include/linux/sysctl.h +2 -0
@@ -344,6 +344,8 @@NET_TCP_DEFAULT_WIN_SCALE =105 ,NET_TCP_MODERATE_RCVBUF =106 ,NET_TCP_TSO_WIN_DIVISOR =107 , +NET_TCP_HYBLA =108 , + NET_TCP_HYBLA_RTT0=109, }; enum {
include/linux/tcp.h +10 -0
@@ -434,6 +434,16 @@__u32 last_cwnd;/* the last snd_cwnd */ __u32 last_stamp;/* time when updated last_cwnd */ } bictcp; + +/* Tcp Hybla structure. */ +struct { +__u32 snd_cwnd_cents;/* Keeps increment values when it is <1, <<7 */ +__u32 rho;/* Rho parameter, integer part */ +__u32 rho2;/* Rho * Rho, integer part */ +__u32 rho_3ls;/* Rho parameter, <<3 */ +__u32 rho2_7ls;/* Rho^2, <<7 */ +__u32 minrtt;/* Minimum smoothed round trip time value seen */ + } hybla; };static inline struct tcp_sock *tcp_sk (const struct sock *sk)
include/net/tcp.h +35 -0
@@ -608,6 +608,8 @@extern int sysctl_tcp_bic_low_window;extern int sysctl_tcp_moderate_rcvbuf;extern int sysctl_tcp_tso_win_divisor; +extern int sysctl_tcp_hybla; +extern int sysctl_tcp_hybla_rtt0;extern atomic_t tcp_memory_allocated;extern atomic_t tcp_sockets_allocated;
@@ -2017,4 +2019,37 @@ return (cwnd != 0); } + +/* +* TCP HYBLA Functions and constants +*/ +/* Hybla reference round trip time (default= 1/40 sec = 25 ms), expressed in jiffies */ +#define RTT0 (__u32) ((HZ/sysctl_tcp_hybla_rtt0)) +/* +* This is called in tcp_ipv4.c and +* tcp_minisocks.c when connection starts +*/ +static inline void init_hybla (struct tcp_sock *tp) +{ + tp->hybla.rho =0 ; + tp->hybla.rho2 =0 ; + tp->hybla.rho_3ls =0 ; + tp->hybla.rho2_7ls =0 ; + tp->hybla.snd_cwnd_cents =0 ; +} +/* This is called to refresh values for hybla parameters */ +static inline void hybla_recalc_param (struct tcp_sock *tp) +{ + + tp->hybla.rho_3ls = (tp->srtt /RTT0 ); +if (tp->hybla.rho_3ls <8 ) + tp->hybla.rho_3ls =8 ; + + tp->hybla.rho=(tp->hybla.rho_3ls >>3 ); + tp->hybla.rho2_7ls = ((tp->hybla.rho_3ls * tp->hybla.rho_3ls)<<1 ); + tp->hybla.rho2=(tp->hybla.rho2_7ls >>7 ); + +if (sysctl_tcp_hybla) + tp->snd_cwnd_clamp =min_t (__u32, tp->snd_cwnd_clamp, tp->hybla.rho<<16 ); +}#endif /* _TCP_H */
net/ipv4/sysctl_net_ipv4.c +16 -0
@@ -682,6 +682,22 @@ .mode = 0644, .proc_handler = &proc_dointvec, }, + { + .ctl_name = NET_TCP_HYBLA, + .procname = "tcp_hybla", + .data = &sysctl_tcp_hybla, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &proc_dointvec + }, + { + .ctl_name = NET_TCP_HYBLA_RTT0, + .procname = "tcp_hybla_rtt0", + .data = &sysctl_tcp_hybla_rtt0, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &proc_dointvec + }, { .ctl_name = 0 } };
net/ipv4/tcp.c +4 -0
@@ -1813,6 +1813,10 @@if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK ))inet_reset_saddr (sk); + +/* [TCP HYBLA] Reset values on disconnect */ +if (sysctl_tcp_hybla) +init_hybla (tp); sk->sk_shutdown =0 ;sock_reset_flag (sk,SOCK_DONE );
net/ipv4/tcp_input.c +177 -3
219 lines not on this page · raw message
net/ipv4/tcp_ipv4.c +3 -0
@@ -2055,6 +2055,9 @@* efficiently to them. -DaveM */ tp->snd_cwnd =2 ; + +/* Reset hybla parameters on socket initialization. */ +init_hybla (tp);/* See draft-stevens-tcpca-spec-01 for discussion of the * initialization of these values.
net/ipv4/tcp_minisocks.c +3 -0
@@ -784,6 +784,9 @@ newtp->dsack =0 ; newtp->eff_sacks =0 ; + +/* Reset hybla parameters on socket initialization. */ +init_hybla (newtp); newtp->probes_out =0 ; newtp->num_sacks =0 ;
Quoted 12 lines · On Tue, 22 Feb 2005 15:34:42 +0100 Daniele Lacamera <[email protected]>…
Hi
This is the official patch to implement TCP Hybla congestion avoidance.- "In heterogeneous networks, TCP connections that incorporate a terrestrial or satellite radio link are greatly disadvantaged with respect to entirely wired connections, because of their longer round trip times (RTTs). To cope with this problem, a new TCP proposal, the TCP Hybla, is presented and discussed in the paper[1]. It stems from an analytical evaluation of the congestion window dynamics in the TCP standard versions (Tahoe, Reno, NewReno), which suggests the necessary modifications to remove the performance dependence on RTT.[...]"[1]
Quoted 4 lines
[1]: Carlo Caini, Rosario Firrincieli, "TCP Hybla: a TCP enhancement for heterogeneous networks",
International Journal of Satellite Communications and Networking Volume 22, Issue 5 , Pages 547 - 566. September 2004.
This looks interesting.
Quoted 10 lines
Please note that this patch:
- cleanly compiled against 2.6.11-rc4;- reached very good results in terms of fairness and friendliness with other TCP standards and proposals during tests;
- gives a throughput very close to maximum fair share when satellite connections fight for bandwidth in a terrestrial bottleneck, and improves their performance also in case of no congestion;
Quoted 5 lines
- acts exactly as newreno when minimum rtt estimated by the sender is lower or equal to rtt0 (default=25 ms).
The protocol is obviously disabled by default and can be turned on by switching its sysctl, as usual.
Probably the best long term solution is to make the protocol choice be a property of the destination cache.
Quoted 9 lines
As Mr.Yee-Ting Li pointed out in his last thread, the deployments of these experimental protocols are too premature and they should be switched OFF by default to prevent undesirable consequences to network stability.
One last note: IMHO we really need a better way to select congestion avoidance scheme between those available, instead of switching each one on and off. I.e., we can't say how vegas and westwood perform when switched on together, can we?
The protocol choices are mutually exclusive, if you walk through the code (or do experiments), you find that that only one gets used. As part of the longer term plan, I would like to:
- have one sysctl
- choice by route and destination
- union for fields in control block
Is there interest in setting up a semi official "-tcp" tree to hold these? because it might not be of wide interest or stability to be ready for mainline kernel.
Quoted 16 lines · Stephen Hemminger
Quoted 4 lines · On Tue, 22 Feb 2005 15:34:42 +0100 Daniele Lacamera <[email protected]>…
One last note: IMHO we really need a better way to select congestion avoidance scheme between those available, instead of switching each one on and off. I.e., we can't say how vegas and westwood perform when switched on together, can we?
The protocol choices are mutually exclusive, if you walk through the code (or do experiments), you find that that only one gets used. As part of the longer term plan, I would like to:
- have one sysctl
- choice by route and destination
- union for fields in control block
Quoted 3 lines
Is there interest in setting up a semi official "-tcp" tree to hold these? because it might not be of wide interest or stability to be ready for mainline kernel.
An idea I've been toying with for a while now is completely abstracting congestion control. Then you could have congestion control loadable modules, which would avoid this mess of experimental algorithms inside the main-line kernel. If done right, they might be able to work seamlessly with SCTP, too. The tricky part is making sure the interface is complete enough.
-John
David S. Miller
maintainerQuoted 6 lines · On Tue, 22 Feb 2005 13:03:11 -0500 (EST) John Heffner <[email protected]> wrote:
An idea I've been toying with for a while now is completely abstracting congestion control. Then you could have congestion control loadable modules, which would avoid this mess of experimental algorithms inside the main-line kernel. If done right, they might be able to work seamlessly with SCTP, too. The tricky part is making sure the interface is complete enough.
The symbols exported to allow this would need to be EXPORT_SYMBOL_GPL().
Quoted 11 lines · David S. Miller
Quoted 6 lines · On Tue, 22 Feb 2005 13:03:11 -0500 (EST) John Heffner <[email protected]> wrote:
An idea I've been toying with for a while now is completely abstracting congestion control. Then you could have congestion control loadable modules, which would avoid this mess of experimental algorithms inside the main-line kernel. If done right, they might be able to work seamlessly with SCTP, too. The tricky part is making sure the interface is complete enough.
The symbols exported to allow this would need to be
EXPORT_SYMBOL_GPL().
Why's that?
Signature · 2 lines
[ Tobias DiPasquale ] 0x636f6465736c696e67657240676d61696c2e636f6d
Quoted 6 lines · David S. Miller
Quoted 5 lines
Quoted 4 lines
On Tue, 22 Feb 2005 13:03:11 -0500 (EST)
John Heffner <[email protected]> wrote:
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
An idea I've been toying with for a while now is completely abstracting congestion control. Then you could have congestion control loadable modules, which would avoid this mess of experimental algorithms inside the main-line kernel. If done right, they might be able to work seamlessly with SCTP, too. The tricky part is making sure the interface is complete enough.
There might be a noticeable performance impact to making it truly modular. Calling a function in a module is slower. In some tests, I see a 5 to 10% drop in performance when Ethernet driver is a module versus builtin.
You might want to look at how the I/O schedulers are configured as an example.
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
The symbols exported to allow this would need to be
EXPORT_SYMBOL_GPL().Why's that?
Because the kernel developers who hold the collective copyright on the existing GPL TCP code do not want some vendor producing a closed source binary module of "enhanced TCP".
David S. Miller
maintainerQuoted 6 lines · On Tue, 22 Feb 2005 09:42:19 -0800 Stephen Hemminger <[email protected]>…
The protocol choices are mutually exclusive, if you walk through the code (or do experiments), you find that that only one gets used. As part of the longer term plan, I would like to:
- have one sysctl
- choice by route and destination
- union for fields in control block
Let's take a first baby step and make the congestion control algorithm a single enumeration instead of all of this tp->foo_cong_ctrl_on, tp->bar_cong_ctrl_on stuff. Then make the union to shrink the tcp_sock size, we could even use an anonymous union to make the patch a lot smaller.
We can't just get rid of all the existing sysctls. We can add a new one that just makes the choice as you describe. We could therefore do something like this:
enum tcp_congctrl_alg tcp_global_congctrl_alg;
And then we use a special sysctl handler for all the individual sysctl_tcp_bic et al. enablers that does something like:
if (tcp_global_congctrl_alg == TCP_CONGCTRL_BIC) return 1; return 0;
and setting just does the necessary assignment to tcp_global_congctrl_alg. Well... I hope the sysctl framework allows something like this :-) If not, it should not be hard to add.
Quoted 10 lines · Stephen Hemminger
Quoted 1 line · On Tue, 22 Feb 2005 15:34:42 +0100 Daniele Lacamera <[email protected]>…
One last note: IMHO we really need a better way to select congestion avoidance scheme between those available, instead of switching each one on and off. I.e., we can't say how vegas and westwood perform when switched on together, can we?
The protocol choices are mutually exclusive, if you walk through the code (or do experiments), you find that that only one gets used. As part of the longer term plan, I would like to:
- have one sysctl
- choice by route and destination
- union for fields in control block
I'm currently working on a patch to make it a single sysctl, I've got it working (as in, the kernel doesn't crash). I still need to validate the actual implementation.
I'd say the next stage is to merge fields as much as possible.
I doubt the real use of selection by route/dest, all of the high-speed protocols (except possibly for TCP-Hybla) are intended for sender-only servers who push lots of data and should work in all cases and alongside
Reno TCP traffic without undue unfairness.
I hope to finish the clean-up and preparation of H-TCP for inclusion in the kernel and can then help with the unionisation.
Baruch
Quoted 2 lines · On Tue, 22 Feb 2005 09:42:19 -0800 Stephen Hemminger <[email protected]>…
Probably the best long term solution is to make the protocol choice be a property of the destination cache
[..]
Quoted 6 lines
The protocol choices are mutually exclusive, if you walk through the code (or do experiments), you find that that only one gets used. As part of the longer term plan, I would like to:
- have one sysctl
- choice by route and destination
- union for fields in control block
I think it should be nice if we substitute in the struct tcp_opt all the structs related to the congestion control algorithms with a more generic one which could be initialized when the connection is open by reading just one sysctl which defines the algorithm to be used. This could allow later even a user-space application to set the congestion control it desires to use.
Quoted 3 lines
Is there interest in setting up a semi official "-tcp" tree to hold these? because it might not be of wide interest or stability to be ready for mainline kernel.
I perfectly agree.
--
Angelo Dell'Aera 'buffer'
Antifork Research, Inc. http://buffer.antifork.org
Metro Olografix
Quoted 12 lines · Daniele Lacamera
Hi
This is the official patch to implement TCP Hybla congestion avoidance.- "In heterogeneous networks, TCP connections that incorporate a terrestrial or satellite radio link are greatly disadvantaged with respect to entirely wired connections, because of their longer round trip times (RTTs). To cope with this problem, a new TCP proposal, the TCP Hybla, is presented and discussed in the paper[1]. It stems from an analytical evaluation of the congestion window dynamics in the TCP standard versions (Tahoe, Reno, NewReno), which suggests the necessary modifications to remove the performance dependence on RTT.[...]"[1]
Quoted 4 lines
[1]: Carlo Caini, Rosario Firrincieli, "TCP Hybla: a TCP enhancement for heterogeneous networks",
International Journal of Satellite Communications and Networking Volume 22, Issue 5 , Pages 547 - 566. September 2004.
It's disappointing that this paper appears to be available only through subscription sources. If I'm mistaken, please post a URL.
By comparison, papers on Reno, Vegas, Westwood, BicTCP, not to mention just about every other contribution to the field of open Internet protocols has been readily available on the net since the birth of FTP servers.
Signature · 1 line
Mathematics is the supreme nostalgia of our time.
Quoted 3 lines · Matt Mackall
It's disappointing that this paper appears to be available only through subscription sources. If I'm mistaken, please post a URL.
The authors are making a webpage with info and documentation. I suppose that article you are referring has been published, then rights go with the editor.
Signature · 2 lines
Daniele Lacamera root at danielinux.net