Skip to contentSkip to conversation

route with different metric

LARTC Christian Lox 5 participants 12 messages Sep 26, 2001 – Sep 27, 2001

lore
Share & tools
lore
#0 lore raw

Hi everyone!

Sorry if this is a FAQ but I did not find a searchable archive.

What I am planning to do:
2 different connections to 2 different ISPs.
One is handled by an ISDN card, one via ethernet to their router (DSL).
We want to use the DSL line; only if this fails we need the ISDN line.
I played around with the metrics but could not get it to work.

Any hint appreciated,
Christian

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

#1 lore raw
Quoted 11 lines · Christian Lox

Hi everyone!

Sorry if this is a FAQ but I did not find a searchable archive.

What I am planning to do:
2 different connections to 2 different ISPs.
One is handled by an ISDN card, one via ethernet to their router (DSL).
We want to use the DSL line; only if this fails we need the ISDN line.
I played around with the metrics but could not get it to work.

This *will* be an FAQ soon! It's by far the most asked question. The short answer is: you are out of luck. There is no way to reliably automate this.

The best solution found so far is to check in a cron script which connection is up, and in case of problems, run something like 'ip route replace default via 1.2.3.4 dev ppp0' or 'ip route replace default via 4.3.2.1 dev eth1'.

There are more interesting solutions but they are not easy. There is something called 'dead gateway detection'. You may experiment with:

# ip route add default nexthop 1.2.3.4 dev ppp0 nexthop 4.3.2.1 dev eth1

If that works as intended, please let us know!

Regards,

bert hubert

Signature · 8 lines
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
#2 lore raw
In reply to Christian Lox
Quoted 11 lines · On Wed, 26 Sep 2001 15:10:35 +0200 "Christian Lox" <[email protected]> wrote:

Hi everyone!

Sorry if this is a FAQ but I did not find a searchable archive.

What I am planning to do:
2 different connections to 2 different ISPs.
One is handled by an ISDN card, one via ethernet to their router (DSL).
We want to use the DSL line; only if this fails we need the ISDN line.
I played around with the metrics but could not get it to work.

I'm trying to find a solution for this very situation since a long time, but wasn't able to do so. What is working for me now is a small daemon I wrote which essentially replaces ipup and ipdown.

Initially it reads the standard /etc/network/interfaces files, paying attention to an additional parameter of priority which is commented out. Then, the interfaces are set up the normal way, but the routes are set with mask /32 just to reach the gateway and no default route. Then, every 60 seconds, a single ping is sent to the gateway of each of the external interfaces, marking it either up or down. After that, the priority value is checked and the program chooses that with the highest priority. If that is also marked active, nothing else is done, but if it wasn't active, the default route is put to that interface and the firewall script is reinitialized. Of course, if the active interface becomes down, the default route is removed.

This works fine, but it's a real pitty that I can't use both interfaces in a balanced way as this would effectively duplicate my bandwidth while both are up. Another (small) problem with this approach is, that switching interfaces automatically stops operators from realizing that a line is not working for to take corrective action. So I decided to send an email in such case, which of course fails if both interfaces are down because I'm geografically far from these networks. But then, operators do realize that something is wrong.

I wrote this program some days ago and it's really a dirty hack, so I doubt the code would be more useful than this description.

I don't have experience with Cisco routers, but was told that load balancing to two or more independent external links is possible with them. So I guess, somehow it should also be possible with Linux. If somebody could shed light on this, I would be really greatful.

--
Christoph Simon
[email protected]
---
^X^C
q
quit
:q
^C
end
x
exit
ZZ
^D
?
help
NO CARRIER
.

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

#3 lore raw
In reply to Christian Lox
Quoted 17 lines · bert hubert
Quoted 11 lines · Christian Lox

Hi everyone!

Sorry if this is a FAQ but I did not find a searchable archive.

What I am planning to do:
2 different connections to 2 different ISPs.
One is handled by an ISDN card, one via ethernet to their router (DSL).
We want to use the DSL line; only if this fails we need the ISDN line.
I played around with the metrics but could not get it to work.

This *will* be an FAQ soon! It's by far the most asked question. The short answer is: you are out of luck. There is no way to reliably automate this.

Quoted 8 lines

The best solution found so far is to check in a cron script which connection is up, and in case of problems, run something like 'ip route replace default via 1.2.3.4 dev ppp0' or 'ip route replace default via 4.3.2.1 dev eth1'.

There are more interesting solutions but they are not easy. There is something called 'dead gateway detection'. You may experiment with:

# ip route add default nexthop 1.2.3.4 dev ppp0 nexthop 4.3.2.1 dev eth1

The syntax is:
ip route add default nexthop via 1.2.3.4 dev ppp0 nexthop via 4.3.2.1 dev eth1

Quoted 1 line

If that works as intended, please let us know!

I just tried and it does some funny things - still investigating...

Juri

Signature · 8 lines
  If each of us have one object, and we exchange them,
     then each of us still has one object.
  If each of us have one idea,   and we exchange them,
     then each of us now has two ideas.

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
#4 lore raw
In reply to Christian Lox
Quoted 9 lines · Juri Haberland
Quoted 6 lines · bert hubert

There are more interesting solutions but they are not easy. There is something called 'dead gateway detection'. You may experiment with:

# ip route add default nexthop 1.2.3.4 dev ppp0 nexthop 4.3.2.1 dev eth1
Quoted 7 lines

The syntax is:
ip route add default nexthop via 1.2.3.4 dev ppp0 nexthop via 4.3.2.1 dev eth1

Quoted 1 line

If that works as intended, please let us know!

I just tried and it does some funny things - still investigating...

Ok, seems to work, but it looks like you have to have the default routes in the order of the interfaces, e.g.:
eth0: 10.0.1.2 (/24)
eth1: 10.0.2.2 (/24)
ip route add default nexthop via 10.0.1.1 dev eth0 nexthop via 10.0.2.1 dev eth1

Cheers,
Juri

Signature · 8 lines
  If each of us have one object, and we exchange them,
     then each of us still has one object.
  If each of us have one idea,   and we exchange them,
     then each of us now has two ideas.

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
#5 lore raw
In reply to Christian Lox
Quoted 10 lines · Juri Haberland
Quoted 3 lines
Quoted 1 line

If that works as intended, please let us know!

I just tried and it does some funny things - still investigating...

Ok, seems to work, but it looks like you have to have the default routes in the order of the interfaces, e.g.:
eth0: 10.0.1.2 (/24)
eth1: 10.0.2.2 (/24)
ip route add default nexthop via 10.0.1.1 dev eth0 nexthop via 10.0.2.1 dev eth1

I think that that's very unlikely. The important thing is: is this 'sticky'? Is a single TCP session mated to a single route automatically? Or does MASQ do this for you?

I've started writing a chapter on this subject.

Regards,

bert

Signature · 8 lines
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
#6 lore raw
In reply to Christian Lox
Quoted 14 lines · bert hubert
Quoted 10 lines · Juri Haberland
Quoted 3 lines
Quoted 1 line

If that works as intended, please let us know!

I just tried and it does some funny things - still investigating...

Ok, seems to work, but it looks like you have to have the default routes in the order of the interfaces, e.g.:
eth0: 10.0.1.2 (/24)
eth1: 10.0.2.2 (/24)
ip route add default nexthop via 10.0.1.1 dev eth0 nexthop via 10.0.2.1 dev eth1

Quoted 3 lines

I think that that's very unlikely. The important thing is: is this 'sticky'? Is a single TCP session mated to a single route automatically? Or does MASQ do this for you?

I just did a short test with ping. Ping went out through eth0. Then I issued a 'ifconfig eth0 down' while ping was running. It went out eth1...

Juri

Signature · 8 lines
  If each of us have one object, and we exchange them,
     then each of us still has one object.
  If each of us have one idea,   and we exchange them,
     then each of us now has two ideas.

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
#7 lore raw
In reply to Christian Lox
Quoted 1 line · Juri Haberland

bert hubert wrote:

[...]

Quoted 7 lines
Quoted 3 lines

I think that that's very unlikely. The important thing is: is this 'sticky'? Is a single TCP session mated to a single route automatically? Or does MASQ do this for you?

I just did a short test with ping. Ping went out through eth0. Then I issued a 'ifconfig eth0 down' while ping was running. It went out eth1...

I thought this sounded good, but I'm still concerned about what Bert raises.

If this machine is also running daemons, does it send replies back out the same interface they came in on? That'll break a lot of things if you receive packets via ISP1, and send replies back via ISP2.

I still don't know how to handle that properly.

--
Adrian Chung (adrian at enfusion-group dot com)
http://www.enfusion-group.com/~adrian
GPG Fingerprint: C620 C8EA 86BA 79CC 384C E7BE A10C 353B 919D 1A17 [rogue.enfusion-group.com] 2:04pm up 55 days, 5:14, 3 users

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

#8 lore raw
In reply to Christian Lox
Quoted 3 lines · Adrian Chung

If this machine is also running daemons, does it send replies back out the same interface they came in on? That'll break a lot of things if you receive packets via ISP1, and send replies back via ISP2.

That sometimes happens - policy routing can help with that problem:

$ ip rule ls
0:	from all lookup local 
32765:	from 213.156.3.162 lookup pine 
32766:	from all lookup main 
32767:	from all lookup default 
$ ip route ls
195.190.241.81 dev ppp0  proto kernel  scope link  src 194.109.252.152 
213.156.3.160/28 dev eth1  proto kernel  scope link  src 213.156.3.162 
10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.150 
192.168.0.0/24 dev eth2  proto kernel  scope link  src 192.168.0.1 
default via 195.190.241.81 dev ppp0 
$ ip route ls table pine
default via 213.156.3.161 dev eth1 

This forces packets to go the right way.

Regards,

bert

Signature · 8 lines
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
#9 lore raw
In reply to Christian Lox
Quoted 15 lines · bert hubert

That sometimes happens - policy routing can help with that problem:

$ ip rule ls
0:	from all lookup local 
32765:	from 213.156.3.162 lookup pine 
32766:	from all lookup main 
32767:	from all lookup default 
$ ip route ls
195.190.241.81 dev ppp0  proto kernel  scope link  src 194.109.252.152 
213.156.3.160/28 dev eth1  proto kernel  scope link  src 213.156.3.162 
10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.150 
192.168.0.0/24 dev eth2  proto kernel  scope link  src 192.168.0.1 
default via 195.190.241.81 dev ppp0 
Quoted 4 lines
$ ip route ls table pine
default via 213.156.3.161 dev eth1 

This forces packets to go the right way.

Ah! I knew there must have been a way... Thanks!

--
Adrian Chung (adrian at enfusion-group dot com)
http://www.enfusion-group.com/~adrian
GPG Fingerprint: C620 C8EA 86BA 79CC 384C E7BE A10C 353B 919D 1A17 [rogue.enfusion-group.com] 5:12pm up 55 days, 8:22, 4 users

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

#10 lore raw
In reply to Christian Lox
Quoted 22 lines · bert hubert
Quoted 3 lines · Adrian Chung

If this machine is also running daemons, does it send replies back out the same interface they came in on? That'll break a lot of things if you receive packets via ISP1, and send replies back via ISP2.

That sometimes happens - policy routing can help with that problem:

$ ip rule ls
0:      from all lookup local
32765:  from 213.156.3.162 lookup pine
32766:  from all lookup main
32767:  from all lookup default
$ ip route ls
195.190.241.81 dev ppp0  proto kernel  scope link  src 194.109.252.152
213.156.3.160/28 dev eth1  proto kernel  scope link  src 213.156.3.162
10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.150
192.168.0.0/24 dev eth2  proto kernel  scope link  src 192.168.0.1
default via 195.190.241.81 dev ppp0
Quoted 2 lines
$ ip route ls table pine
default via 213.156.3.161 dev eth1

Don't you need all the interface specific routes also in table 'pine'? It seems to me that if you don't have them, all packets from 213.156.3.162 would go out eth1, even those destined to 192.168.0.0/24 (which should go out eth2 in your example)...

Juri

Signature · 8 lines
  If each of us have one object, and we exchange them,
     then each of us still has one object.
  If each of us have one idea,   and we exchange them,
     then each of us now has two ideas.

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
#11 lore raw
In reply to Christian Lox
Quoted 3 lines · Juri Haberland
Quoted 2 lines · bert hubert
Quoted 2 lines

ip route add default nexthop via 10.0.1.1 dev eth0 nexthop via 10.0.2.1 dev eth1

[...]

Quoted 3 lines

I just did a short test with ping. Ping went out through eth0. Then I issued a 'ifconfig eth0 down' while ping was running. It went out eth1...

Although I tried this, and it doesn't work if the connection's dead between your NIC and the ISP, but the interface is still up.

I unplugged my network cable, and did a ping after adding a line with two nexthops as above, and it didn't route through the second link at all.

Then again, I might be doing something wrong. :)

--
Adrian Chung (adrian at enfusion-group dot com)
http://www.enfusion-group.com/~adrian
GPG Fingerprint: C620 C8EA 86BA 79CC 384C E7BE A10C 353B 919D 1A17 [rogue.enfusion-group.com] 6:36pm up 56 days, 9:45, 3 users

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/