2 Internet connection and one local network

DISCUSSION Techside Security 3 participants 6 messages Jan 9, 2007 – Jan 10, 2007

Share & tools

Techside Security

author

2007-01-09 08:27 UTC #0 lore raw

Hi, this configuration don't work for me.
I have traced the packet that arrive from internet to a internal server;
when the packet is sended to public ip that corrspond to default internet line all is ok, but when i send a packet to public ip that correspond to second internet line the packet arrive to firewall and don't go forward to internal server. This seems to be an nat or forwarding error but I if add the table rule (iproute2)
ip rule add from <internal server ip> table line2
the packet go to server and return from the second line.
All the test is made with the iptables and iproute rule
described in the reply post.

What is the meaning of: echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter

I'm using debian sarge with 2.6.17 kernel and iptables 1.3.7

Any suggestion on what I'm wrong ?

Sorry for my bad english.
Fabio.

Quoted 174 lines

Date: Thu, 4 Jan 2007 11:08:42 -0000
From: "Matt" <[email protected]>
Subject: RE: 2 Internet connection and one local network
To: <[email protected]>
Message-ID:
	<117F5E7DA31C17478948DC39E01B948B400F91@frost.PlumSoftwareLtd.local>
Content-Type: text/plain; charset="ISO-8859-15"
Copied from an earlier post of mine... Similiar situation to yours, I believe. Hope you can make sense of it all...
Matt.
Copied text below:
For your, and anyone else's interest, the answer is:
====================== IPTABLES commands =================================
#!/bin/sh
IPTABLES="/sbin/iptables"
# prevent incoming packets on masqueraded connections from being dropped
# as "martians" due to the destination address being translated before the
# rp_filter check is performed
# MATT NOTES: this does not seem to be necessary...
#echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter
#echo 0 > /proc/sys/net/ipv4/conf/eth2/rp_filter
#Time to clean house
#Clear out any existing firewall rules, and any chains that might have #been created
$IPTABLES -F
$IPTABLES -F INPUT
$IPTABLES -F OUTPUT
$IPTABLES -F FORWARD
$IPTABLES -F -t mangle
$IPTABLES -F -t nat
$IPTABLES -X
#Setup our policies
$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -P FORWARD ACCEPT
#This enables ip forwarding, and thus by extension, NAT echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/modprobe ip_conntrack
/sbin/modprobe ip_conntrack_ftp
/sbin/modprobe ip_nat_ftp
/sbin/modprobe iptable_nat
#Our actual rules
#Our NAT stuff
#Source NAT everything heading out the external interface to be the #given IP. $IPTABLES -t nat -A POSTROUTING -o eth1 -j SNAT --to 100.100.251.218 $IPTABLES -t nat -A POSTROUTING -o eth2 -j SNAT --to 200.200.64.140
#Mark incoming packets for later routing
$IPTABLES -t mangle -A PREROUTING -j CONNMARK --restore-mark $IPTABLES -t mangle -A PREROUTING -i eth1 -j MARK --set-mark 11 $IPTABLES -t mangle -A PREROUTING -i eth2 -j MARK --set-mark 12
#save mark on outgoing packets
$IPTABLES -t mangle -A POSTROUTING -j CONNMARK --save-mark
#These are the rules for publishing the internal server
$IPTABLES -t nat -A PREROUTING -i eth1 -p tcp --dport 56100 -j DNAT --to 192.168.0.5 $IPTABLES -t nat -A PREROUTING -i eth2 -p tcp --dport 56100 -j DNAT --to 192.168.0.5
================== IPROUTE2 STUFF ===========================
# Set up supplementary routing tables
ip route add 100.100.251.216/29 dev eth1 src 100.100.251.218 table T1 ip route add default via 100.100.251.217 table T1
ip route add 200.200.64.136/29 dev eth2 src 200.200.64.140 table T2 ip route add default via 200.200.64.137 table T2
# Not necessary as these routes are added by configuring network cards #ip route add 100.100.251.216/29 dev eth1 src 100.100.251.218 #ip route add 200.200.64.136/29 dev eth2 src 200.200.64.140
# set default route for traffic originating from this machine to go via third router
# not necessarily what other people may want to do, as you may want outgoing traffic
# going through eth1/eth2 in some load balanced way
ip route add default via 192.168.0.252
ip rule add from 100.100.251.218 table T1
ip rule add from 200.200.64.140 table T2
#not sure what this does, but is recommended in Advanced Routing HOWTO ip route add 192.168.0.0/24 dev eth0 table T1
ip route add 200.200.64.136/29 dev eth2 table T1
ip route add 127.0.0.0/8 dev lo table T1
ip route add 192.168.0.0/24 dev eth0 table T2
ip route add 100.100.251.216/29 dev eth1 table T2
ip route add 127.0.0.0/8 dev lo table T2
#route based on mark
ip rule add fwmark 11 table T1
ip rule add fwmark 12 table T2
================================================
Regards,
Matt.
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Techside Security
Sent: 04 January 2007 09:37
To: [email protected]
Subject: 2 Internet connection and one local network
Hi,
in my company I have to switch to another provider and for a while I have to maintain both the internet connection.
I have 2 server that is reachable from internet that is natted 1:1 with local ip.
INTERNET OLD (32 public ip) INTERNET NEW (32 public ip)
	|			|
	|			|
	-------------------------
	|			|
	|	FIREWALL	|
	|			|
	-------------------------
		|
		|
		LAN (private network)
Now, the firewall nat all the connection from static ip to local ip for the 2 server (WEB, MAIL) and for the navigation of the client.
public ip 1 -----> FRW ------> local mail server ip public ip 2 -----> FRW ------> local web server ip
client navigation ------> FRW --------> default public ip (! mail,! web)
The problem is that I can't let to be reachable the server from both internet connection. Now work only for one connection (with iproute2 and rules) based on the source local ip of the server.
I have tried with MARK, CONNMARK, CONNTRACK but I'm confused about it. There is a way to make this working ?
All info are very appreciated.
Thanks.

Matt

2007-01-10 09:37 UTC #1 lore raw

Hi.

The default gateway on your internal servers should point to the internal IP of the linux firewall box. It sounds to me that you've set the default gateway to the public IP of one of your internet lines - doing this will certainly stop it from working.

This configuration should remember what internet line the packet arrived at, and when the reply from the internal server arrives back at the linux box, it should be routed back out the same internet line it arrived at.

Hope that helps,

Matt

-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Techside Security
Sent: 09 January 2007 08:28
To: [email protected]
Subject: RE: 2 Internet connection and one local network

Hi, this configuration don't work for me.
I have traced the packet that arrive from internet to a internal server; when the packet is sended to public ip that corrspond to default internet line all is ok, but when i send a packet to public ip that correspond to second internet line the packet arrive to firewall and don't go forward to internal server. This seems to be an nat or forwarding error but I if add the table rule (iproute2)
ip rule add from <internal server ip> table line2
the packet go to server and return from the second line.
All the test is made with the iptables and iproute rule described in the reply post.

What is the meaning of: echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter

I'm using debian sarge with 2.6.17 kernel and iptables 1.3.7

Any suggestion on what I'm wrong ?

Sorry for my bad english.
Fabio.

Quoted 175 lines

Date: Thu, 4 Jan 2007 11:08:42 -0000
From: "Matt" <[email protected]>
Subject: RE: 2 Internet connection and one local network
To: <[email protected]>
Message-ID:
	<117F5E7DA31C17478948DC39E01B948B400F91@frost.PlumSoftwareLtd.local>
Content-Type: text/plain; charset="ISO-8859-15"
Copied from an earlier post of mine... Similiar situation to yours, I believe. Hope you can make sense of it all...
Matt.
Copied text below:
For your, and anyone else's interest, the answer is:
====================== IPTABLES commands =================================
#!/bin/sh
IPTABLES="/sbin/iptables"
# prevent incoming packets on masqueraded connections from being dropped
# as "martians" due to the destination address being translated before the
# rp_filter check is performed
# MATT NOTES: this does not seem to be necessary...
#echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter
#echo 0 > /proc/sys/net/ipv4/conf/eth2/rp_filter
#Time to clean house
#Clear out any existing firewall rules, and any chains that might have #been created
$IPTABLES -F
$IPTABLES -F INPUT
$IPTABLES -F OUTPUT
$IPTABLES -F FORWARD
$IPTABLES -F -t mangle
$IPTABLES -F -t nat
$IPTABLES -X
#Setup our policies
$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -P FORWARD ACCEPT
#This enables ip forwarding, and thus by extension, NAT echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/modprobe ip_conntrack
/sbin/modprobe ip_conntrack_ftp
/sbin/modprobe ip_nat_ftp
/sbin/modprobe iptable_nat
#Our actual rules
#Our NAT stuff
#Source NAT everything heading out the external interface to be the #given IP.
$IPTABLES -t nat -A POSTROUTING -o eth1 -j SNAT --to 100.100.251.218 $IPTABLES -t nat -A POSTROUTING -o eth2 -j SNAT --to 200.200.64.140
#Mark incoming packets for later routing
$IPTABLES -t mangle -A PREROUTING -j CONNMARK --restore-mark $IPTABLES -t mangle -A PREROUTING -i eth1 -j MARK --set-mark 11 $IPTABLES -t mangle -A PREROUTING -i eth2 -j MARK --set-mark 12
#save mark on outgoing packets
$IPTABLES -t mangle -A POSTROUTING -j CONNMARK --save-mark
#These are the rules for publishing the internal server
$IPTABLES -t nat -A PREROUTING -i eth1 -p tcp --dport 56100 -j DNAT --to 192.168.0.5 $IPTABLES -t nat -A PREROUTING -i eth2 -p tcp --dport 56100 -j DNAT --to 192.168.0.5
================== IPROUTE2 STUFF ===========================
# Set up supplementary routing tables
ip route add 100.100.251.216/29 dev eth1 src 100.100.251.218 table T1 ip route add default via 100.100.251.217 table T1
ip route add 200.200.64.136/29 dev eth2 src 200.200.64.140 table T2 ip route add default via 200.200.64.137 table T2
# Not necessary as these routes are added by configuring network cards #ip route add 100.100.251.216/29 dev eth1 src 100.100.251.218 #ip route add 200.200.64.136/29 dev eth2 src 200.200.64.140
# set default route for traffic originating from this machine to go via third router
# not necessarily what other people may want to do, as you may want outgoing traffic
# going through eth1/eth2 in some load balanced way
ip route add default via 192.168.0.252
ip rule add from 100.100.251.218 table T1
ip rule add from 200.200.64.140 table T2
#not sure what this does, but is recommended in Advanced Routing HOWTO ip route add 192.168.0.0/24 dev eth0 table T1
ip route add 200.200.64.136/29 dev eth2 table T1
ip route add 127.0.0.0/8 dev lo table T1
ip route add 192.168.0.0/24 dev eth0 table T2
ip route add 100.100.251.216/29 dev eth1 table T2
ip route add 127.0.0.0/8 dev lo table T2
#route based on mark
ip rule add fwmark 11 table T1
ip rule add fwmark 12 table T2
================================================
Regards,
Matt.
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Techside Security
Sent: 04 January 2007 09:37
To: [email protected]
Subject: 2 Internet connection and one local network
Hi,
in my company I have to switch to another provider and for a while I have to maintain both the internet connection.
I have 2 server that is reachable from internet that is natted 1:1 with local ip.
INTERNET OLD (32 public ip) INTERNET NEW (32 public ip)
	|			|
	|			|
	-------------------------
	|			|
	|	FIREWALL	|
	|			|
	-------------------------
		|
		|
		LAN (private network)
Now, the firewall nat all the connection from static ip to local ip for the 2 server (WEB, MAIL) and for the navigation of the client.
public ip 1 -----> FRW ------> local mail server ip public ip 2 -----> FRW ------> local web server ip
client navigation ------> FRW --------> default public ip (! mail,! web)
The problem is that I can't let to be reachable the server from both internet connection. Now work only for one connection (with iproute2 and rules) based on the source local ip of the server.
I have tried with MARK, CONNMARK, CONNTRACK but I'm confused about it. There is a way to make this working ?
All info are very appreciated.
Thanks.

ArioS

2007-01-10 10:10 UTC #2 lore raw

anyone can tell me what`r these ?

I got it few time ago and it down all of my wireless networking. is it DoS ? what can of dos ? i`m using linux as router.. any chance to block it ?

Iptraf :

Non-IP (0x4) (1500 bytes) from ffffffff0000 to ffffffffffff on eth0 Non-IP (0x4) (1500 bytes) from ffffffff0000 to ffffffffffff on eth0 Non-IP (0x4) (1500 bytes) from ffffffff0000 to ffffffffffff on eth0 Non-IP (0x4) (1500 bytes) from ffffffff0000 to ffffffffffff on eth0

TcpDump :

15:11:19.765769 ff:ff:ff:ff:00:00 > Broadcast null I (s=0,r=0,C) len=42 15:11:19.766271 ff:ff:ff:ff:00:00 > Broadcast null I (s=0,r=0,C) len=42 15:11:19.767359 ff:ff:ff:ff:00:00 > Broadcast null I (s=0,r=0,C) len=42 15:11:19.770860 ff:ff:ff:ff:00:00 > Broadcast null I (s=0,r=0,C) len=42

Techside Security

author

2007-01-10 14:43 UTC #3 lore raw

In reply to Matt

Hi.

Unfortunately all server have set the default gw to the internal ip of the firewall... With tcpdump seems that the packet don't reach the internal server but reach the external ethernet of the firewall (this sounds very strange....)

ping rqst -----> PUBLIC IP1 ----> |FRW| --\ \--- internal srv (this don't work)

ping rqst -----> PUBLIC IP2 (default gw) ----> |FRW| ----> internal srv (this work)

My configuration is little different about yours; i nat entire server IP
not only port from the 2 isp lines.

Now the only way to use the new line for all the service (except sigh the servers) is to setup the default gw to the new router and to maintain the old line with the server (now all the service use this old public ip) whith the rule:

ip rule add from <internalsrvip> table oldline

If I delete this rule the server respond to the new line but not respond to the call from the old line.

thks.

Matt ha scritto:

Quoted 38 lines

Hi.
The default gateway on your internal servers should point to the internal IP of the linux firewall box. It sounds to me that you've set the default gateway to the public IP of one of your internet lines - doing this will certainly stop it from working.
This configuration should remember what internet line the packet arrived at, and when the reply from the internal server arrives back at the linux box, it should be routed back out the same internet line it arrived at.
Hope that helps,
Matt
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Techside Security
Sent: 09 January 2007 08:28
To: [email protected]
Subject: RE: 2 Internet connection and one local network
Hi, this configuration don't work for me.
I have traced the packet that arrive from internet to a internal server;
when the packet is sended to public ip that corrspond to default internet line all is ok, but when i send a packet to public ip that correspond to second internet line the packet arrive to firewall and don't go forward to internal server. This seems to be an nat or forwarding error but I if add the table rule (iproute2)
ip rule add from <internal server ip> table line2
the packet go to server and return from the second line.
All the test is made with the iptables and iproute rule
described in the reply post.
What is the meaning of: echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter
I'm using debian sarge with 2.6.17 kernel and iptables 1.3.7
Any suggestion on what I'm wrong ?
Sorry for my bad english.
Fabio.

Matt

2007-01-10 14:39 UTC #4 lore raw

In reply to Techside Security

Sorry I missed this part:

Quoted 1 line

Quoted 1 line
Quoted 1 line
What is the meaning of: echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter

I think it disables the reverse path filter for that interface (eth1). Now you might ask, what is the reverse path filter? And I don't know! Apparently it does this:

Quoted 3 lines

# prevent incoming packets on masqueraded connections from being dropped
# as "martians" due to the destination address being translated before the
# rp_filter check is performed

However, when I commented out the line it made no difference so I've left it there for people to comment back in if they find they have problems as described.

Regards,

Matt.

Techside Security

author

2007-01-10 15:44 UTC #5 lore raw

Re: 2 Internet connection and one local network -- RESOLVED

Ok, i've tried to use it because it seems that firewall don't understand the packet coming from old line (without rule).... the result i great!!!

Now all works, I have setup the rule for all the two ethernet card of the internet lines.

echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth2/rp_filter

Thanks a lot!!

Matt ha scritto:

Quoted 21 lines

Sorry I missed this part:
Quoted 1 line
Quoted 1 line
Quoted 1 line
What is the meaning of: echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter
I think it disables the reverse path filter for that interface (eth1). Now you might ask, what is the reverse path filter? And I don't know! Apparently it does this:
Quoted 3 lines
# prevent incoming packets on masqueraded connections from being dropped
# as "martians" due to the destination address being translated before the
# rp_filter check is performed
However, when I commented out the line it made no difference so I've left it there for people to comment back in if they find they have problems as described.
Regards,
Matt.