Strange routing limitations and workaroud
Hi!
I got some strange problem with routing loadbalancing.
I cannot get the full speed from my ISPs until I get some big files from close ftp server.
I have server with one connection to internal network and 3 to ISPs:
__________
| eth1|---- ISP1
| |
internal--|eth0 eth2|---- ISP2
net | |
(~300 | eth3|---- ISP3hosts |_________|
I have done everything as described in LARTC, chapter 4.2
The main rule looks like that (the weight reflects link speed/100):
ip route add default scope global \
nexthop via $ISP1_GATEWAY dev eth1 weight 12 \
nexthop via $ISP2_GATEWAY dev eth2 weight 10 \
nexthop via $ISP3_SPRINT_GATEWAY dev eth3 weight 20Total bandwidth available is 4.2 Mbit.
After I restart the server I can get 2.0Mbit maximum, with first link 5% utilized, and the second link and the third about 50%.
When I get some big files from close ftp server (4 x linux kernel = ~80MB) the links start to work normal, reaching 75-100% utilization. All those big files go via the first link.
Can someone possibly explain that and teach me how to get full speed without such shamanism?
Szymon Miotk
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 31 lines · Szymon Miotk
Hi!
I got some strange problem with routing loadbalancing.
I cannot get the full speed from my ISPs until I get some big files from close ftp server.
I have server with one connection to internal network and 3 to ISPs:__________ | eth1|---- ISP1 | | internal--|eth0 eth2|---- ISP2 net | | (~300 | eth3|---- ISP3 hosts |_________|I have done everything as described in LARTC, chapter 4.2
The main rule looks like that (the weight reflects link speed/100):
ip route add default scope global \nexthop via $ISP1_GATEWAY dev eth1 weight 12 \ nexthop via $ISP2_GATEWAY dev eth2 weight 10 \ nexthop via $ISP3_SPRINT_GATEWAY dev eth3 weight 20Total bandwidth available is 4.2 Mbit.
After I restart the server I can get 2.0Mbit maximum, with first link 5% utilized, and the second link and the third about 50%.
When I get some big files from close ftp server (4 x linux kernel = ~80MB) the links start to work normal, reaching 75-100% utilization. All those big files go via the first link.
Quoted 4 lines
Can someone possibly explain that and teach me how to get full speed without such shamanism?
Szymon Miotk
"This balancing will not be perfect."
Try this:www.linuxvirtualserver.org/~julian/nano.txt
should be better.
Signature · 9 lines
GPG-Key: http://www.bartek.bicom.pl/public_key.txt pub 1024D/948DE45D 2002-12-12 Bartek Krajnik <[email protected]> Primary key fingerprint: 95E9 8E2D 1801 7864 2244 6EAA 03E5 764D 948D E45D _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[sorry for the delay; I had postponed the mail and forgotten to send]
Szymon,
You'll want to read this thread from a month ago (or so):
http://mailman.ds9a.nl/pipermail/lartc/2003q1/006781.html
To summarize, packets bound for the a destination in the routing cache will be routed out according to the cached entry. One ugly, hackish, and very dumb solution is to expire the route cache frequently.
Note that I have not done this! Others on the list have suggested the "equalize" route parameter as a solution for this problem. For the operation as you desire, you'll need to patch your kernel with Patrick McHardy's equalize patch, available here:
http://trash.net/~kaber/
http://trash.net/~kaber/equalize_2.4.18.patch
Good luck,
-Martin
: I got some strange problem with routing loadbalancing.
: I cannot get the full speed from my ISPs until I get some big files from : close ftp server.
: I have server with one connection to internal network and 3 to ISPs:
:
:
: __________
: | eth1|---- ISP1 : | | : internal--|eth0 eth2|---- ISP2 : net | | : (~300 | eth3|---- ISP3 : hosts |_________| : : I have done everything as described in LARTC, chapter 4.2 : : The main rule looks like that (the weight reflects link speed/100): : ip route add default scope global \ : nexthop via $ISP1_GATEWAY dev eth1 weight 12 \ : nexthop via $ISP2_GATEWAY dev eth2 weight 10 \ : nexthop via $ISP3_SPRINT_GATEWAY dev eth3 weight 20
:
: Total bandwidth available is 4.2 Mbit.
: After I restart the server I can get 2.0Mbit maximum, with first link 5%
: utilized, and the second link and the third about 50%.
: When I get some big files from close ftp server (4 x linux kernel : ~80MB) the links start to work normal, reaching 75-100% utilization. All : those big files go via the first link.
:
: Can someone possibly explain that and teach me how to get full speed
: without such shamanism?
:
: Szymon Miotk
:
: _______________________________________________
: LARTC mailing list / [email protected]
: http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
:
Signature · 5 lines
Martin A. Brown --- SecurePipe, Inc. --- [email protected] _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/