Skip to contentSkip to conversation

Strange routing limitations and workaroud

LARTC Szymon Miotk 3 participants 3 messages Feb 6, 2003 – Feb 13, 2003

lore
Share & tools
lore
#0 lore raw

Hi!

I got some strange problem with routing loadbalancing.
I cannot get the full speed from my ISPs until I get some big files from close ftp server.
I have server with one connection to internal network and 3 to ISPs:

            __________
           |     eth1|---- ISP1
           |         |
internal--|eth0 eth2|---- ISP2
net       |         |
(~300     |     eth3|---- ISP3

hosts |_________|

I have done everything as described in LARTC, chapter 4.2

The main rule looks like that (the weight reflects link speed/100):
ip route add default scope global \

        nexthop via $ISP1_GATEWAY dev eth1 weight 12 \
        nexthop via $ISP2_GATEWAY dev eth2 weight 10 \
        nexthop via $ISP3_SPRINT_GATEWAY dev eth3 weight 20

Total bandwidth available is 4.2 Mbit.
After I restart the server I can get 2.0Mbit maximum, with first link 5% utilized, and the second link and the third about 50%.
When I get some big files from close ftp server (4 x linux kernel = ~80MB) the links start to work normal, reaching 75-100% utilization. All those big files go via the first link.

Can someone possibly explain that and teach me how to get full speed without such shamanism?

Szymon Miotk

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#1 lore raw
Quoted 31 lines · Szymon Miotk

Hi!

I got some strange problem with routing loadbalancing.
I cannot get the full speed from my ISPs until I get some big files from close ftp server.
I have server with one connection to internal network and 3 to ISPs:

           __________
          |     eth1|---- ISP1
          |         |
internal--|eth0 eth2|---- ISP2
net       |         |
(~300     |     eth3|---- ISP3
 hosts    |_________|

I have done everything as described in LARTC, chapter 4.2

The main rule looks like that (the weight reflects link speed/100):
ip route add default scope global \

       nexthop via $ISP1_GATEWAY dev eth1 weight 12 \
       nexthop via $ISP2_GATEWAY dev eth2 weight 10 \
       nexthop via $ISP3_SPRINT_GATEWAY dev eth3 weight 20

Total bandwidth available is 4.2 Mbit.
After I restart the server I can get 2.0Mbit maximum, with first link 5% utilized, and the second link and the third about 50%.
When I get some big files from close ftp server (4 x linux kernel = ~80MB) the links start to work normal, reaching 75-100% utilization. All those big files go via the first link.

Quoted 4 lines

Can someone possibly explain that and teach me how to get full speed without such shamanism?

Szymon Miotk

"This balancing will not be perfect."
Try this:
www.linuxvirtualserver.org/~julian/nano.txt
should be better.

Signature · 9 lines
GPG-Key:
   http://www.bartek.bicom.pl/public_key.txt
   pub  1024D/948DE45D 2002-12-12 Bartek Krajnik <[email protected]>
   Primary key fingerprint: 95E9 8E2D 1801 7864 2244  6EAA 03E5 764D 948D E45D


_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#2 lore raw
In reply to Szymon Miotk

[sorry for the delay; I had postponed the mail and forgotten to send]

Szymon,

You'll want to read this thread from a month ago (or so):

http://mailman.ds9a.nl/pipermail/lartc/2003q1/006781.html

To summarize, packets bound for the a destination in the routing cache will be routed out according to the cached entry. One ugly, hackish, and very dumb solution is to expire the route cache frequently.

Note that I have not done this! Others on the list have suggested the "equalize" route parameter as a solution for this problem. For the operation as you desire, you'll need to patch your kernel with Patrick McHardy's equalize patch, available here:

http://trash.net/~kaber/
http://trash.net/~kaber/equalize_2.4.18.patch

Good luck,

-Martin

: I got some strange problem with routing loadbalancing.
: I cannot get the full speed from my ISPs until I get some big files from : close ftp server.
: I have server with one connection to internal network and 3 to ISPs:
:
:
: __________

 :            |     eth1|---- ISP1
 :            |         |
 :  internal--|eth0 eth2|---- ISP2
 :  net       |         |
 :  (~300     |     eth3|---- ISP3
 :   hosts    |_________|
 :
 : I have done everything as described in LARTC, chapter 4.2
 :
 : The main rule looks like that (the weight reflects link speed/100):
 : ip route add default scope global \
 :         nexthop via $ISP1_GATEWAY dev eth1 weight 12 \
 :         nexthop via $ISP2_GATEWAY dev eth2 weight 10 \
 :         nexthop via $ISP3_SPRINT_GATEWAY dev eth3 weight 20

:
: Total bandwidth available is 4.2 Mbit.
: After I restart the server I can get 2.0Mbit maximum, with first link 5%
: utilized, and the second link and the third about 50%.
: When I get some big files from close ftp server (4 x linux kernel : ~80MB) the links start to work normal, reaching 75-100% utilization. All : those big files go via the first link.
:
: Can someone possibly explain that and teach me how to get full speed
: without such shamanism?
:
: Szymon Miotk
:
: _______________________________________________
: LARTC mailing list / [email protected]
: http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
:

Signature · 5 lines
Martin A. Brown --- SecurePipe, Inc. --- [email protected]

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/