Skip to contentSkip to conversation

Need information on multi-homing

LARTC Bruce Perens 2 participants 5 messages Mar 2, 2002 – Mar 4, 2002

lore
Share & tools
lore
#0 lore raw

Is there a multi-homing FAQ? I've not been able to find it.

Here's my system's configuration:

	eth0:	SDSL at 216.15.108.186 (dnai-net)
	eth1:	ADSL at 67.114.175.138 (sbc-net)
	eth2:	DHCP network at 192.168.1.1 .
	lo:	127.0.0.1

	There is a firewall box acting as the DHCP server and router for the
	systems on the DHCP net. The Linux system doesn't run DHCP or
	masquerading.

If a connection is accepted on the IP address of one of these interfaces, I'd like its packets to go out on the same interface. If there is a failure on the path to eth0, connections to eth1 should continue to work, and vice versa. But right now, eth0 has the default route and just about every packet goes out of eth0 regardless of what interface they come in. So, I tried this:

	ip rule add from 216.15.108.186 table dnai-net 
	ip rule add from 67.114.175.138 table sbc-net
	ip route add default via 216.15.108.186 dev eth0 table dnai-net
	ip route add default via 67.114.175.138 dev eth1 table sbc-net

This doesn't work. I get a destination unreachable if I try to follow either route. So, what am I doing wrong?

Thanks

Bruce Perens
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#1 lore raw
Quoted 1 line · Bruce Perens

Is there a multi-homing FAQ? I've not been able to find it.

No, but it is very high on my agenda. Multihoming and loadbalancing are among the most popular subjects if I count by my inbox.

Quoted 10 lines

Here's my system's configuration:

	eth0:	SDSL at 216.15.108.186 (dnai-net)
	eth1:	ADSL at 67.114.175.138 (sbc-net)
	eth2:	DHCP network at 192.168.1.1 .
	lo:	127.0.0.1

	There is a firewall box acting as the DHCP server and router for the
	systems on the DHCP net. The Linux system doesn't run DHCP or
	masquerading.

So what IP address do packets have that come from the firewall box?

Quoted 6 lines

If a connection is accepted on the IP address of one of these interfaces, I'd like its packets to go out on the same interface. If there is a failure on the path to eth0, connections to eth1 should continue to work, and vice versa. But right now, eth0 has the default route and just about every packet goes out of eth0 regardless of what interface they come in. So, I tried this:

Quoted 7 lines
	ip rule add from 216.15.108.186 table dnai-net 
	ip rule add from 67.114.175.138 table sbc-net
	ip route add default via 216.15.108.186 dev eth0 table dnai-net
	ip route add default via 67.114.175.138 dev eth1 table sbc-net

This doesn't work. I get a destination unreachable if I try to follow either route. So, what am I doing wrong?

You have no default route left for packets which came in over the firewall box, which probably has a 192.168.1.x address? I think that is the number one thing going wrong.

Now, as to what you want, you will need a way to separate connections to go out over either eth0 or eth1 in a persistent way. This mostly means sending out half your internal IP range over one interface and the other half over the other.

This in turn implies SNAT or masquerading because you need to change the source address - packets with the address of eth0 won't travel the net well over eth1.

Regards,

bert hubert

Signature · 6 lines
http://www.PowerDNS.com          Versatile DNS Software & Services
http://www.tk                              the dot in .tk
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#2 lore raw
In reply to Bruce Perens

Bert Hubert:

Quoted 1 line

So what IP address do packets have that come from the firewall box?

Oops, I didn't give enough information. The Linux box is not a router - it's a multi-homed server. The DHCP net has one of those retail router boxes that does DHCP, NAT, and is a gateway to one of the DSL networks.

If I try to follow a policy route to either of the DSL interfaces from the Linux system itself, not _through_ it as a router.

	ip rule add from 216.15.108.186 table dnai-net 
	ip rule add from 67.114.175.138 table sbc-net
	ip route add default via 216.15.108.186 dev eth0 table dnai-net
	ip route add default via 67.114.175.138 dev eth1 table sbc-net

	ping -I 67.114.175.138 www.gnu.org

I get an IMAP "unreachable" message back from the interface.

What I want is to be able to bind to the IP address of either of the DSL interfaces, and have the packets routed to that interface reliably. If there's a network partition effecting one DSL carrier and not the other, it should still be possible to reach the system and the return packets should be on the same network as the incoming ones.

Load-balancing is secondary, but I have some ideas there. For example, why not have squid alternate addresses so that receive data is interleaved across the two DSL lines? Wouldn't that balance better than simply routing half of the internet through each interface?

Thanks

Bruce
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#3 lore raw
In reply to Bruce Perens
Quoted 6 lines · Bruce Perens

Bert Hubert:

Quoted 1 line

So what IP address do packets have that come from the firewall box?

Oops, I didn't give enough information. The Linux box is not a router - it's a multi-homed server. The DHCP net has one of those retail router boxes that does DHCP, NAT, and is a gateway to one of the DSL networks.

Ah ok, so we have the following:

	eth0 SDSL
        ---  
         |     
       Linux -------- [router appliance] ---- ADSL -internet
         |    eth1  
        ---
        eth2 DHCP
Quoted 11 lines

If I try to follow a policy route to either of the DSL interfaces from the Linux system itself, not _through_ it as a router.

	ip rule add from 216.15.108.186 table dnai-net 
	ip rule add from 67.114.175.138 table sbc-net
	ip route add default via 216.15.108.186 dev eth0 table dnai-net
	ip route add default via 67.114.175.138 dev eth1 table sbc-net

	ping -I 67.114.175.138 www.gnu.org

I get an IMAP "unreachable" message back from the interface.

I still lack information - what are the IP addresses of the Linux machine? Right now you are telling the kernel to route packets FROM 216.15.108.186 TO 216.15.108.186, which isn't happening :-)

Quoted 5 lines

What I want is to be able to bind to the IP address of either of the DSL interfaces, and have the packets routed to that interface reliably. If there's a network partition effecting one DSL carrier and not the other, it should still be possible to reach the system and the return packets should be on the same network as the incoming ones.

You should make sure the policy routes are made from the IP address of eth0 and eth1, and go to the addresses of the routers there. In that case, ping -I should work. Right now you aren't routing anything to anything, leading to an ICMP unreachable.

Quoted 4 lines

Load-balancing is secondary, but I have some ideas there. For example, why not have squid alternate addresses so that receive data is interleaved across the two DSL lines? Wouldn't that balance better than simply routing half of the internet through each interface?

It isn't persistent. Many internet sites rely on the fact that users keep coming from the same IP address during their session.

Regards,

bert

Signature · 6 lines
http://www.PowerDNS.com          Versatile DNS Software & Services
http://www.tk                              the dot in .tk
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#4 lore raw
In reply to Bruce Perens
Quoted 2 lines · bert hubert

Right now you are telling the kernel to route packets FROM 216.15.108.186 TO 216.15.108.186, which isn't happening :-)

I should have known better than _that_. OK, it works now. Thanks very much.

Quoted 6 lines
Quoted 3 lines

why not have squid alternate addresses so that receive data is interleaved across the two DSL lines? Wouldn't that balance better than simply routing half of the internet through each interface?

It isn't persistent. Many internet sites rely on the fact that users keep coming from the same IP address during their session.

Good point. I don't know how, or if, proxies like squid use persistent connections. I suspect that some load-balancing schemes already break them. apt-get (Debian's package downloader) already knows how to parallelize hosts. It would be nice to parallelize the interfaces at the same time.

FYI, this is a picture of my net.

	768K SDSL ------------------------------------ eth0


	1.5M..384K/128K ADSL ------------------------- eth1
					^
					| (10MBPS)
					|
				Retail Router Box		Linux Box
				Serves local DHCP
				hosts with DHCP, NAT,
				"firewall", gateway.
					^
					|
					|
        Local DHCP net ------------------------------- eth2

All networks are 100/10 MBit full-duplex switches. Both DSL modems and the uplink port of the retail router run 10 MBit half-duplex, everything else runs 100 MBit full-duplex. Not much should be going through the retail router box, as the Linux box serves the local net with squid, IMAP, SMTP, and telnet. I could dispense with the router box and route with Linux if I wanted to.

Attached is what I ended up with in /etc/network/interfaces:

Thanks

Bruce

auto eth0
iface eth0 inet static

    address 216.15.108.186
    network 216.15.108.184
    netmask 255.255.255.248
    broadcast 216.15.108.255
    # gateway 216.15.108.185
    up ip rule add from 216.15.108.186 table dnai-net 
    up ip route add default via 216.15.108.185 dev eth0 table dnai-net
    up ip route add default via 216.15.108.185 dev eth0 metric 1 table main
    down ip rule del from 216.15.108.186 table dnai-net 
    down ip route del default via 216.15.108.185 dev eth0 table dnai-net
    down ip route del default via 216.15.108.185 dev eth0 metric 1 table main

auto eth1
iface eth1 inet static

    address 67.114.175.138
    network 67.114.175.137
    netmask 255.255.255.248
    broadcast 67.114.175.143
    up ip rule add from 67.114.175.138 table sbc-net
    up ip route add default via 67.114.175.137 dev eth1 table sbc-net
    up ip route add default via 67.114.175.137 dev eth1 metric 2 table main
    down ip rule del from 67.114.175.138 table sbc-net
    down ip route del default via 67.114.175.137 dev eth1 table sbc-net
    down ip route del default via 67.114.175.137 dev eth1 metric 2 table main

auto eth2
iface eth2 inet static

    address 192.168.1.1
    network 192.168.1.0
    netmask 255.255.255.0
    broadcast 192.168.1.255

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/