Skip to contentSkip to conversation

LoadBalancing on many asimetric different dsl's.

LARTC sAwAr 3 participants 6 messages Jan 22, 2007 – Jan 23, 2007

lore
Share & tools
lore

sAwAr

author
#0 lore raw

Hi,

my company have just bought new network and I have question about one problem.
As in topic we must use few completely different dsl's and balance traffic between them.
2M/0,5M 4Mb/0,5M 8M/0,5M
M=Mb/s
I've never done such thing before so I have doubts how it will work. If the links are symmetric 2/2 4/4 8/8 there is no problem because with weights I can compensate the difference between them and achieve nice results. But what in my situation? My questions are: how to set load balancing to get all links equally loaded and avoid situation when the up load will be full and download almost empty? I believe this situation can happen due to fact that load balancing is based on flows and for example p2p or smpt/pop3 will eat whole upload. If my problem isn't clear I'll try to explain it better later.

Thanks in advance.
Pozdrawiam
sawar

----------------------------------------------------------------------
Wolne adresy pocztowe @interia.eu >>> http://link.interia.pl/f19e8

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

#1 lore raw

Hello,

I've done this some montsh ago, with a command like:
ip route add default equalize scope global nexthop via $EXTGW1 dev $EXTIF1 weight 1 nexthop via $EXTGW2 dev $EXTIF2 weight 1

However, this is not the problem.
While loadbalancing of simple requests worked fine, there where problems when you worked with connections. I mean HTTPS, of FTP connection for example.

The problem was fo me that the system trys to send packets of the same connection throught different gateways, so with different IP source (each DSL connection was from different ISP). This caused the server not to understand why the same connection sent packets with 2 different source IP ;)
Well, I hope you understand me.

If you would do real load balancing, and in a proper way, you should not only do it by link charge, but route packets by connection to. (routing all packets of the same connection through the same gateway) This is caused because you must flush the route cache some times (or packets to a destination will allways take the same route, wich is not a loadbalance).

So if someone has done it and doesn't have this problem, I'm interested too :)

Thanks!

Jordi Segues

Quoted 23 lines · sAwAr

Hi,

my company have just bought new network and I have question about one problem.
As in topic we must use few completely different dsl's and balance traffic between them.
2M/0,5M 4Mb/0,5M 8M/0,5M
M=Mb/s
I've never done such thing before so I have doubts how it will work. If the links are symmetric 2/2 4/4 8/8 there is no problem because with weights I can compensate the difference between them and achieve nice results. But what in my situation? My questions are: how to set load balancing to get all links equally loaded and avoid situation when the up load will be full and download almost empty? I believe this situation can happen due to fact that load balancing is based on flows and for example p2p or smpt/pop3 will eat whole upload. If my problem isn't clear I'll try to explain it better later.

Thanks in advance.
Pozdrawiam
sawar

----------------------------------------------------------------------
Wolne adresy pocztowe @interia.eu >>> http://link.interia.pl/f19e8

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Signature · 13 lines
Jordi Segués Daina
-------------------------------------------------------
Andorra GSM: (+376) 35 35 68
France GSM: (+33) (0)6 81 88 35 55
E-m@il / MSN: [email protected]
AIM: superjordix
Skype: callto://superjordix
-------------------------------------------------------
http://www.JordiX.com
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
#2 lore raw
In reply to sAwAr
Quoted 25 lines · Jordi Segues

Hello,

I've done this some montsh ago, with a command like:
ip route add default equalize scope global nexthop via $EXTGW1 dev $EXTIF1 weight 1 nexthop via $EXTGW2 dev $EXTIF2 weight 1

However, this is not the problem.
While loadbalancing of simple requests worked fine, there where problems when you worked with connections. I mean HTTPS, of FTP connection for example.

The problem was fo me that the system trys to send packets of the same connection throught different gateways, so with different IP source (each DSL connection was from different ISP). This caused the server not to understand why the same connection sent packets with 2 different source IP ;)
Well, I hope you understand me.

If you would do real load balancing, and in a proper way, you should not only do it by link charge, but route packets by connection to. (routing all packets of the same connection through the same gateway) This is caused because you must flush the route cache some times (or packets to a destination will allways take the same route, wich is not a loadbalance).

Quoted 2 lines

So if someone has done it and doesn't have this problem, I'm interested too :)

the above is actually covered in the wiki howto. Bu tyou need to setup snat on each interface, then connection tracking takes care of sending each stream out the right interface, you need to use snat and not MASQ.

Then you need to setup up some ip rule tables for each of the interfaces.

my ip ru looks like this

0:      from all lookup local 
200:    from 144.132.145.38 lookup cable 
201:    from 60.241.248.86 lookup adsl 
32766:  from all lookup main 
32767:  from all lookup default 

my ip r sh tab default

default proto static metric 5

        nexthop via 144.132.144.1  dev vlan2 weight 1
        nexthop via 10.20.20.230  dev ppp0 weight 20

default via 10.20.20.230 dev ppp0 src 60.241.248.86 metric 20 default via 144.132.144.1 dev vlan2 src 144.132.145.38 metric 30

This works fine for me, I have tracked packets with tcpdump on both the server and the client.

Alex

Quoted 30 lines

Thanks!

Jordi Segues

Quoted 24 lines · sAwAr

Hi,

my company have just bought new network and I have question about one problem.
As in topic we must use few completely different dsl's and balance traffic between them.
2M/0,5M 4Mb/0,5M 8M/0,5M
M=Mb/s
I've never done such thing before so I have doubts how it will work. If the links are symmetric 2/2 4/4 8/8 there is no problem because with weights I can compensate the difference between them and achieve nice results. But what in my situation?
My questions are: how to set load balancing to get all links equally loaded and avoid situation when the up load will be full and download almost empty? I believe this situation can happen due to fact that load balancing is based on flows and for example p2p or smpt/pop3 will eat whole upload.
If my problem isn't clear I'll try to explain it better later.

Thanks in advance.
Pozdrawiam
sawar

Quoted 25 lines
Quoted 8 lines

----------------------------------------------------------------------
Wolne adresy pocztowe @interia.eu >>> http://link.interia.pl/f19e8

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Signature · 13 lines
Jordi Segués Daina
-------------------------------------------------------
Andorra GSM: (+376) 35 35 68
France GSM: (+33) (0)6 81 88 35 55
E-m@il / MSN: [email protected]
AIM: superjordix
Skype: callto://superjordix
-------------------------------------------------------
http://www.JordiX.com
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

#3 lore raw
In reply to sAwAr
Quoted 3 lines

the above is actually covered in the wiki howto. Bu tyou need to setup snat on each interface, then connection tracking takes care of sending each stream out the right interface, you need to use snat and not MASQ.

Great news :)
And thankyou for the details.
But could you give the link to the wiki howto?
I only found old doc.

Thanks!

Quoted 101 lines

Then you need to setup up some ip rule tables for each of the interfaces.

my ip ru looks like this

0:      from all lookup local
200:    from 144.132.145.38 lookup cable
201:    from 60.241.248.86 lookup adsl
32766:  from all lookup main
32767:  from all lookup default

my ip r sh tab default

default proto static metric 5

        nexthop via 144.132.144.1  dev vlan2 weight 1
        nexthop via 10.20.20.230  dev ppp0 weight 20

default via 10.20.20.230 dev ppp0 src 60.241.248.86 metric 20 default via 144.132.144.1 dev vlan2 src 144.132.145.38 metric 30

This works fine for me, I have tracked packets with tcpdump on both the server and the client.

Alex

Quoted 40 lines

Thanks!

Jordi Segues

Quoted 24 lines · sAwAr

Hi,

my company have just bought new network and I have question about one problem.
As in topic we must use few completely different dsl's and balance traffic between them.
2M/0,5M 4Mb/0,5M 8M/0,5M
M=Mb/s
I've never done such thing before so I have doubts how it will work. If the links are symmetric 2/2 4/4 8/8 there is no problem because with weights I can compensate the difference between them and achieve nice results. But what in my situation?
My questions are: how to set load balancing to get all links equally loaded and avoid situation when the up load will be full and download almost empty? I believe this situation can happen due to fact that load balancing is based on flows and for example p2p or smpt/pop3 will eat whole upload.
If my problem isn't clear I'll try to explain it better later.

Thanks in advance.
Pozdrawiam
sawar

Quoted 8 lines

----------------------------------------------------------------------
Wolne adresy pocztowe @interia.eu >>> http://link.interia.pl/f19e8

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Quoted 15 lines

--
Jordi Segués Daina
-------------------------------------------------------
Andorra GSM: (+376) 35 35 68
France GSM: (+33) (0)6 81 88 35 55
E-m@il / MSN: [email protected]
AIM: superjordix
Skype: callto://superjordix
-------------------------------------------------------
http://www.JordiX.com
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFFtJ1/kZz88chpJ2MRAhGKAJ9xthAZnQ/ovr82sa/x5j4BFJGgWwCgvtWa dS7qseaia3GnZK/n8szE98Y> =zLpL
-----END PGP SIGNATURE-----

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Signature · 13 lines
Jordi Segués Daina
-------------------------------------------------------
Andorra GSM: (+376) 35 35 68
France GSM: (+33) (0)6 81 88 35 55
E-m@il / MSN: [email protected]
AIM: superjordix
Skype: callto://superjordix
-------------------------------------------------------
http://www.JordiX.com
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
#4 lore raw
In reply to sAwAr
Quoted 10 lines · Jordi Segues
Quoted 5 lines

the above is actually covered in the wiki howto. Bu tyou need to setup snat on
each interface, then connection tracking takes care of sending each stream out
the right interface, you need to use snat and not MASQ.

Great news :)
And thankyou for the details.
But could you give the link to the wiki howto?
I only found old doc.

been a while since i had a look, quick google gave me this

http://lartc.org/howto/lartc.rpdb.multiple-links.html

I have this booked market as the wiki
http://linux-net.osdl.org/index.php/Main_Page

But I think the former is what you want

Quoted 100 lines

Thanks!

Quoted 97 lines

Then you need to setup up some ip rule tables for each of the interfaces.

my ip ru looks like this

0:      from all lookup local
200:    from 144.132.145.38 lookup cable
201:    from 60.241.248.86 lookup adsl
32766:  from all lookup main
32767:  from all lookup default

my ip r sh tab default

default proto static metric 5

       nexthop via 144.132.144.1  dev vlan2 weight 1
       nexthop via 10.20.20.230  dev ppp0 weight 20

default via 10.20.20.230 dev ppp0 src 60.241.248.86 metric 20 default via 144.132.144.1 dev vlan2 src 144.132.145.38 metric 30

This works fine for me, I have tracked packets with tcpdump on both the server
and the client.

Alex

Quoted 11 lines

Thanks!

Jordi Segues

Quoted 5 lines · sAwAr

Hi,

my company have just bought new network and I have question about one problem.
As in topic we must use few completely different dsl's and balance

traffic

Quoted 29 lines
Quoted 19 lines

between them.
2M/0,5M 4Mb/0,5M 8M/0,5M
M=Mb/s
I've never done such thing before so I have doubts how it will work. If the links are symmetric 2/2 4/4 8/8 there is no problem because with weights I can compensate the difference between them and achieve nice results. But what in my situation?
My questions are: how to set load balancing to get all links equally loaded and avoid situation when the up load will be full and download almost empty? I believe this situation can happen due to fact that load balancing is based on flows and for example p2p or smpt/pop3 will eat whole upload.
If my problem isn't clear I'll try to explain it better later.

Thanks in advance.
Pozdrawiam
sawar

Quoted 8 lines

----------------------------------------------------------------------
Wolne adresy pocztowe @interia.eu >>> http://link.interia.pl/f19e8

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Quoted 15 lines

--
Jordi Segués Daina
-------------------------------------------------------
Andorra GSM: (+376) 35 35 68
France GSM: (+33) (0)6 81 88 35 55
E-m@il / MSN: [email protected]
AIM: superjordix
Skype: callto://superjordix
-------------------------------------------------------
http://www.JordiX.com
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFFtJ1/kZz88chpJ2MRAhGKAJ9xthAZnQ/ovr82sa/x5j4BFJGgWwCgvtWa dS7qseaia3GnZK/n8szE98Y=
=zLpL
-----END PGP SIGNATURE-----

Quoted 24 lines
Quoted 7 lines

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Signature · 13 lines
Jordi Segués Daina
-------------------------------------------------------
Andorra GSM: (+376) 35 35 68
France GSM: (+33) (0)6 81 88 35 55
E-m@il / MSN: [email protected]
AIM: superjordix
Skype: callto://superjordix
-------------------------------------------------------
http://www.JordiX.com
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

author
#5 lore raw
In reply to sAwAr

Thanks for all your answers. I ask my question in different way because I still don't get answer which will be satysfying for me.
Did anybody set similar configuration in the past? Have someone any suggestions how to set it to configure loadbalancing to avoid situation when one link is empty or only upload is used and other links are full. Does weights ensure that upload and download on all links with different up/down speeds will be equally loaded?

Pozdrawiam
sawar

Quoted 81 lines · Jordi Segues
Quoted 4 lines
Quoted 4 lines

the above is actually covered in the wiki howto. Bu tyou need to setup

snat on
each interface, then connection tracking takes care of sending each

stream

Quoted 7 lines
Quoted 2 lines

out
the right interface, you need to use snat and not MASQ.

Great news :)
And thankyou for the details.
But could you give the link to the wiki howto?
I only found old doc.

been a while since i had a look, quick google gave me this

http://lartc.org/howto/lartc.rpdb.multiple-links.html

I have this booked market as the wiki
http://linux-net.osdl.org/index.php/Main_Page

But I think the former is what you want

Quoted 5 lines

Thanks!

Quoted 2 lines

Then you need to setup up some ip rule tables for each of the

interfaces.

Quoted 30 lines
Quoted 30 lines

my ip ru looks like this

0:      from all lookup local
200:    from 144.132.145.38 lookup cable
201:    from 60.241.248.86 lookup adsl
32766:  from all lookup main
32767:  from all lookup default

my ip r sh tab default

default proto static metric 5

       nexthop via 144.132.144.1  dev vlan2 weight 1
       nexthop via 10.20.20.230  dev ppp0 weight 20

default via 10.20.20.230 dev ppp0 src 60.241.248.86 metric 20 default via 144.132.144.1 dev vlan2 src 144.132.145.38 metric 30

This works fine for me, I have tracked packets with tcpdump on both the

server
and the client.

Alex

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thanks!

Jordi Segues

Quoted 3 lines · On 22 Jan 2007 09:49:28 +0100, sAwAr >[email protected]> wrote:

Hi,

my company have just bought new network and I have question about

one

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

problem.
As in topic we must use few completely different dsl's and balance

traffic

Quoted 4 lines
Quoted 4 lines

between them.
2M/0,5M 4Mb/0,5M 8M/0,5M
M=Mb/s
I've never done such thing before so I have doubts how it will work.

If

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

the links are symmetric 2/2 4/4 8/8 there is no problem because

with

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

weights I can compensate the difference between them and achieve

nice

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

results. But what in my situation?
My questions are: how to set load balancing to get all links

Quoted 77 lines

equally

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

loaded and avoid situation when the up load will be full and

download

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

almost empty? I believe this situation can happen due to fact that

load

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

balancing is based on flows and for example p2p or smpt/pop3 will

eat

Quoted 68 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

whole upload.
If my problem isn't clear I'll try to explain it better later.

Thanks in advance.
Pozdrawiam
sawar

----------------------------------------------------------------------

Quoted 41 lines
Quoted 9 lines
Quoted 7 lines

Wolne adresy pocztowe @interia.eu >>> http://link.interia.pl/f19e8

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Quoted 15 lines

--
Jordi Segués Daina
-------------------------------------------------------
Andorra GSM: (+376) 35 35 68
France GSM: (+33) (0)6 81 88 35 55
E-m@il / MSN: [email protected]
AIM: superjordix
Skype: callto://superjordix
-------------------------------------------------------
http://www.JordiX.com
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFFtJ1/kZz88chpJ2MRAhGKAJ9xthAZnQ/ovr82sa/x5j4BFJGgWwCgvtWa dS7qseaia3GnZK/n8szE98Y> > >=zLpL
-----END PGP SIGNATURE-----

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Signature · 13 lines
Jordi Segués Daina
-------------------------------------------------------
Andorra GSM: (+376) 35 35 68
France GSM: (+33) (0)6 81 88 35 55
E-m@il / MSN: [email protected]
AIM: superjordix
Skype: callto://superjordix
-------------------------------------------------------
http://www.JordiX.com
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

----------------------------------------------------------------------
Co robi Indianin w banku? Zobacz >> http://link.interia.pl/f19e4

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc