Skip to contentSkip to conversation

Help: Multiple internet connections

LARTC David H. Lynch Jr. 2 participants 4 messages Sep 25, 2002 – Sep 27, 2002

lore
Share & tools
lore
#0 lore raw

I have been trying to get a router/firewall with a DSL and 6 fixed
IP's and a Faster Cable modem with a semi-fixed IP up and running for a long, long time.

Mostly things work, and I have read the
http://lartc.org/howto/lartc.rpdb.multiple-links.html#AEN261 pages and implemented them.

I appear to have full access to the extent that my iptables rules
allow to any services running on the router/firewall.

I appear to have outgoing internet access NATed correctly and
running the way I want.

However I have problems with the servers/services that are being
DNATed to behind the firewall.

I do not believe my problems are with IPTABLES - at least I have
forced IPTABLES to log every packet it drops and it is not dropping anything related to the areas I am having problems.

There are a number of odd things that appear to be going on, but for
the most part the failure cases appear to occur when a client on the internet who is using an ISP closely related to one of my connections, tries to connect through an address on the interface farthest away from them.

I.E. when a client also using a local cable connection tries to
connect to a DNATed server/service using a DSL IP.

		My DSL is 209.223.245.120-128
		my cable is 68.84.207,53
	A client whose IP is 68.84.207.97 tries to connect to

209.223.245.125:143 and is unable to connect.

IPTABLES logs no dropped packets. iproute is configured as per
the multiple links pages above.

It is my guess that the inbound packet manages its way to my
server just fine, but on the return trip it decides to head back out the cable modem as that is the best route back to the client, and since the client sees a response coming from the wrong source it discards it, but I could easily be wrong.

I believe I am only having problems with DNATed services behind
the firewall, and I believe it is only when the client is local to the external interface opposite the one they are coming in on. But I could easily be wrong. regardless the problem is most ly reproducible - though it has been know to go away for days at a time on its own, and mostly limited to a small subset of all clients.

I am busily perusing the lartc archives but have not found
anything directly on point yet.

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#1 lore raw
Quoted 3 lines · David H. Lynch Jr.

Mostly things work, and I have read the
http://lartc.org/howto/lartc.rpdb.multiple-links.html#AEN261 pages and implemented them.

*blink*, nice to hear that. :)

Quoted 2 lines

I appear to have full access to the extent that my iptables rules
allow to any services running on the router/firewall.

Good.

Quoted 2 lines

I appear to have outgoing internet access NATed correctly and
running the way I want.

Good.

Quoted 2 lines

However I have problems with the servers/services that are being
DNATed to behind the firewall.

Not so good.

[snip]

Quoted 5 lines

It is my guess that the inbound packet manages its way to my
server just fine, but on the return trip it decides to head back out the cable modem as that is the best route back to the client, and since the client sees a response coming from the wrong source it discards it, but I could easily be wrong.

No, you are most probably right. Unfortunately, there is no real solution to your problem, for as soon as the packet has ben DNATed to the service behind the firewall you lose all information as to which route the packet took to get to your firewall. This means that any return packet can only take the `obvious' short route directly to the remote machine, and not the less-obvious route the long way round but with the correct source address.

Quoted 6 lines

I believe I am only having problems with DNATed services behind
the firewall, and I believe it is only when the client is local to the external interface opposite the one they are coming in on. But I could easily be wrong. regardless the problem is most ly reproducible - though it has been know to go away for days at a time on its own, and mostly limited to a small subset of all clients.

Sounds as if there's routes flapping for a subset of your clients.

I can see only one real solution: have some sort of application-level proxies run on your firewall host to plug the connections through to the services behind it. One way to do so would be to use socks in listening mode. Another would be to use netcat...

Doei, Arthur.

Signature · 7 lines
  /\    / |      [email protected]      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#2 lore raw
In reply to David H. Lynch Jr.
Quoted 10 lines

-----Original Message-----
From: Arthur van Leeuwen [mailto:[email protected]]
Sent: Thursday, September 26, 2002 5:28 AM
To: David H. Lynch Jr.
Cc: 'lartc Mailing List'
Subject: Re: [LARTC] Help: Multiple internet connections

Quoted 8 lines
Quoted 2 lines

However I have problems with the servers/services that are being
DNATed to behind the firewall.

Not so good.

[snip]

Quoted 1 line

It is my guess that the inbound packet manages its way to my

server

Quoted 6 lines
Quoted 4 lines

just fine, but on the return trip it decides to head back out the cable modem as that is the best route back to the client, and since the client sees a response coming from the wrong source it discards it, but I could easily be wrong.

No, you are most probably right. Unfortunately, there is no real

solution to your problem, for as soon as the packet has

Quoted 1 line

ben DNATed to the service behind the firewall you lose all information

as to which route the packet took to get to

Quoted 1 line

your firewall. This means that any return packet can only take the

`obvious' short route directly to the remote machine,

Quoted 1 line

and not the less-obvious route the long way round but with the correct

source address.

Quoted 1 line
Quoted 1 line

I believe I am only having problems with DNATed services behind

the

Quoted 2 lines
Quoted 2 lines

firewall, and I believe it is only when the client is local to the external interface opposite the one they are coming in on. But I

could

Quoted 8 lines
Quoted 4 lines

easily be wrong. regardless the problem is most ly reproducible - though it has been know to go away for days at a time on its own, and

mostly limited to a small subset of all clients.

Sounds as if there's routes flapping for a subset of your clients.

I can see only one real solution: have some sort of application-level

proxies run on your firewall host to plug the

Quoted 1 line

connections through to the services behind it. One way to do so would

be to use socks in listening mode.

Quoted 3 lines

Another would be to use netcat...

Doei, Arthur.

Trying to grok the interrelations between IPTABLES and routing has given me a headache. I guess I am not as sharp as I used to be. I am also having a hard time getting a complete handle on what "stateful" really means. But I am gathering that this is a routing problem caused as a side effect of DNATing a connection. If IPTABLES is "stateful" does that mean that if I MARK a packet that the return packet is also marked ? If that were the case I could mark the Inbound packets from one interface and use iproute to select the right routing table for the return packets. Alternatively, if I set the servers behind the firewall up with two IP's and DNATed to a different one depending on the incoming interface shouldn't I be able to chose an outgoing routing table based on the source IP of the return packet ? Finally what is a flapping route ? This problem would make allot more sense to me if it were consistent.

Thank you. Just having a second opinion that I am on the right
track helps allot.

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#3 lore raw
In reply to David H. Lynch Jr.

On Fri, 27 Sep 2002, David H. Lynch Jr. wrote:

[snip, trying to do multipath routing through a NAT box]

Quoted 6 lines

Trying to grok the interrelations between IPTABLES and routing has given me a headache. I guess I am not as sharp as I used to be. I am also having a hard time getting a complete handle on what "stateful" really means. But I am gathering that this is a routing problem caused as a side effect of DNATing a connection. If IPTABLES is "stateful" does that mean that if I MARK a packet that the return packet is also marked ?

No. It means that the firewall maintains internal state as to what packets it has seen. This allows it to determine that packets from a TCP connection are allowed, due to the fact that it has indeed been setup from this side.

The problem you're seeing is that you let the packets travel out of your firewall again to the server behind it. As soon as the packets are on the wire, there is *no* way to do any tracking of the any more, as you can with fwmarks as long as they stay on the host. The problem is that you lose the information on what interface the packet has originally come in, due to DNAT.

Quoted 4 lines

Alternatively, if I set the servers behind the firewall up with two IP's and DNATed to a different one depending on the incoming interface shouldn't I be able to chose an outgoing routing table based on the source IP of the return packet ?

That is exactly the right solution to this problem, if you don't intend to do it at the application level. :)

Quoted 2 lines

Finally what is a flapping route ? This problem would make allot more sense to me if it were consistent.

Owh, the internet provider of the clients that you see `disappearing' from the server at times re-routes the traffic from time to time. It probably has redundant links to the internet, and using some of these links the problem does occur whereas using others it doesn't. However, due to reconfigurations, hardware failure, all kinds of reasons really, the routes may not be very stable and `flap' between the different links. That is what I meant.

Quoted 2 lines

Thank you. Just having a second opinion that I am on the right
track helps allot.

Well, you are. :)

Doei, Arthur.

Signature · 7 lines
  /\    / |      [email protected]      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/