simple dual Internet connection setup not sending return packets
I have a very simple setup exactly as described in the HOWTO section " 4.2. Routing for multiple uplinks/providers".
One is cable (eth1: dhcp) and the other is PPPoE (ppp0).
I used the following commands to configure the routing once all of my interfaces are up and i have configured SNATing for them:
ip route add 66.11.173.0/24 dev ppp0 src 66.11.173.224 table 11 ip route add default via 66.11.190.1 table 11
ip route add 24.235.240.0/22 dev eth1 src 24.235.240.15 table 12 ip route add default via 24.235.240.1 table 12
ip route add 66.11.173.0/24 dev ppp0 src 66.11.173.224 ip route add 24.235.240.0/22 dev eth1 src 24.235.240.15 ip rule add from 66.11.173.224 table 11
ip rule add from 24.235.240.15 table 12
My iptables "nat" setup looks like this:
Chain POSTROUTING (policy ACCEPT 364 packets, 26735 bytes)
pkts bytes target prot opt in out source destination
258 19801 eth1_masq all -- * eth1 0.0.0.0/0 0.0.0.0/0
0 0 ppp0_masq all -- * ppp0 0.0.0.0/0 0.0.0.0/0Chain eth1_masq (1 references)
pkts bytes target prot opt in out source destination
252 19021 SNAT all -- * * 10.75.22.0/24 0.0.0.0/0 to:24.235.240.15
0 0 SNAT all -- * * 192.168.66.0/24 0.0.0.0/0 to:24.235.240.15Chain ppp0_masq (1 references)
pkts bytes target prot opt in out source destination
0 0 SNAT all -- * * 10.75.22.0/24 0.0.0.0/0 to:66.11.173.224
0 0 SNAT all -- * * 192.168.66.0/24 0.0.0.0/0 to:66.11.173.224When an IP packet enters my machine via interface ppp0 everything is fine. However when a responding packet (i.e. a SYN-ACK in response to a SYN) is sent from my machine, it is sent with the correct source address (66.11.173.224) but on the eth1 interface. It is worth noting at this point that eth1 is the default gateway interface.
Is this a problem in that iproute2 selects the default route before SNATting is done to change the source address of the packet, which does not happen of course until POSTROUTING? Surely I am not the first person who has this need. How does one solve it?
Current routing tables:
# ip route list 66.11.190.1 dev ppp0 proto kernel scope link src 66.11.173.224 66.11.173.0/24 dev ppp0 scope link src 66.11.173.224 10.75.22.0/24 dev eth0 proto kernel scope link src 10.75.22.254 24.235.240.0/22 dev eth1 proto kernel scope link src 24.235.240.15 default via 24.235.240.1 dev eth1
# ip route list table 11 66.11.173.0/24 dev ppp0 scope link src 66.11.173.224 default via 66.11.190.1 dev ppp0
# ip route list table 12 24.235.240.0/22 dev eth1 scope link src 24.235.240.15 default via 24.235.240.1 dev eth1
and current ip rules:
# ip rule list 0: from all lookup local 32764: from 24.235.240.15 lookup 12 32765: from 66.11.173.224 lookup 11 32766: from all lookup main 32767: from all lookup default
Any ideas what I am doing wrong?
b.
Re: [LARTC] simple dual Internet connection setup not sending return
Quoted 6 lines · Brian J. Murrell
I have a very simple setup exactly as described in the HOWTO section " 4.2. Routing for multiple uplinks/providers".
My iptables "nat" setup looks like this:
Quoted 8 lines
Chain POSTROUTING (policy ACCEPT 364 packets, 26735 bytes)
pkts bytes target prot opt in out source destination 258 19801 eth1_masq all -- * eth1 0.0.0.0/0 0.0.0.0/0 0 0 ppp0_masq all -- * ppp0 0.0.0.0/0 0.0.0.0/0Any ideas what I am doing wrong?
b.
Guessing from the lack of any mention of KeepState in your iptables setup, my guess is that you ignored the advice to vist Julian Anastasov's web site.
Start with this:
http://www.geocities.com/mctiew/ffw/dual.htm
You should also google LARTC "Finally: A working case of two adsl load balance". Read Ron Senykoff's post "load balance a file download across two connections - success!".
gypsy
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] simple dual Internet connection setup not sending
In reply to Brian J. MurrellQuoted 2 lines · gypsy
Guessing from the lack of any mention of KeepState
KeepState? If you are referring to:
52459 2774K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
rules, I have those sprinkled throughout my ruleset where necessary. The iptables "snippet" I included in my previous message was just that. Just the relevant portion that does the NATting.
Quoted 2 lines
in your iptables
setup,
Like I said, the RELATED,ESTABLISHED state rules are in there. My full set of iptables rules is >400. I did not see see a need to post that fully here.
Quoted 2 lines
my guess is that you ignored the advice to vist Julian
Anastasov's web site.
No I didn't ignore it. But what that site is promoting is some kind of floppy disk based router distribution or something.
Quoted 3 lines
Start with this:
http://www.geocities.com/mctiew/ffw/dual.htm
I am not looking to replace/rebuild my whole firewall. I simply want to add a second link to my existing one and have the packets use the correct interface -- to travel back out the interface from which they came.
I don't want to do load balancing or failover or anything fancy. I want two interfaces where I use one for all outgoing traffic and the only time the alternate is used is to send response packets to connections that come _in_ that interface or for routes that are specifically directed through that interface via a routing table entry.
Quoted 3 lines
You should also google LARTC "Finally: A working case of two adsl load balance". Read Ron Senykoff's post "load balance a file download across two connections - success!".
Interesting. Followed a few links too. Looks like a lot of bells and whistles I am not really looking for (load balancing and failover, etc.) but there is some hint of indication that there is a patch needed to make sure NAT uses the right physical interface. Maybe I will go bug the netfilter guys to see if this is the case.
Thanx,
b.
Re: [LARTC] simple dual Internet connection setup not sending
In reply to Brian J. MurrellTo followup on my own posting, with more information...
Quoted 4 lines · Brian J. Murrell
I have a very simple setup exactly as described in the HOWTO section " 4.2. Routing for multiple uplinks/providers".
One is cable (eth1: dhcp) and the other is PPPoE (ppp0).
These are both on the same physical interface, eth1. IOW, the PPPoE packets are sent to the PPPoE "modem" on eth1. eth1 is also plugged into the cable provider's "modem" as such:
+---------- Cable Modem
+--------+ |
| | +--+--+
| GW eth1 ------| HUB |
| | +--+--+
+--------+ |
+---------- PPPoE ModemThis set up works, physically. I can tcpdump on eth1 and see both regular ethernet traffic going to an from my cable provider, as well as PPPoE encapsulated traffic coming in through my PPPoE connection:
09:29:58.109041 00:08:e2:33:f8:54 > 00:a0:24:2a:1f:72, ethertype IPv4 (0x0800), length 130: IP 66.96.26.190.922 > 24.235.240.15.52814: P 49:113(64) ack 48 win 28800 <nop,nop,timestamp 59750486 1599607031>
09:29:58.109344 00:a0:24:2a:1f:72 > 00:08:e2:33:f8:54, ethertype IPv4 (0x0800), length 66: IP 24.235.240.15.52814 > 66.96.26.190.922: . ack 113 win 32740 <nop,nop,timestamp 1599607172 59750486>
09:29:58.117164 00:90:1a:40:43:d7 > 00:a0:24:2a:1f:72, ethertype PPPoE S (0x8864), length 82: PPPoE [ses 0x1473] PPP-IP (0x0021), length 62: IP 66.96.26.190.52797 > 66.11.173.224.25: S 3517919246:3517919246(0) win 5840 <mss 1400,sackOK,timestamp 59750486 0,nop,wscale 0> 09:29:58.118789 00:a0:24:2a:1f:72 > 00:08:e2:33:f8:54, ethertype IPv4 (0x0800), length 74: IP 66.11.173.224.25 > 66.96.26.190.52797: S 3862223559:3862223559(0) ack 3517919247 win 5792 <mss 1460,sackOK,timestamp 2207063156 59750486,nop,wscale 0>
As you can see, packets 1 and 2 are an established TCP session over the cable connection and packet 3 is an incoming PPPoE encapsulated packet coming in on the PPPoE connection and interestingly enough, packet 4 is an erroneously transmitted packet demonstrating exactly my problem. It is the response to packet 3. As you can see it has all of the correct IP and TCP headers, it is just sent physically via eth1 and not ppp0.
Heh. Indeed if my cable provider were not filtering packets from me that don't have my assigned source address, this would all work.
Just a reminder of my iptables SNAT rules for context of my point below...
Quoted 12 lines
My iptables "nat" setup looks like this:
Chain POSTROUTING (policy ACCEPT 364 packets, 26735 bytes)
pkts bytes target prot opt in out source destination 258 19801 eth1_masq all -- * eth1 0.0.0.0/0 0.0.0.0/0 0 0 ppp0_masq all -- * ppp0 0.0.0.0/0 0.0.0.0/0Chain
eth1_masq(1 references)pkts bytes target prot opt in out source destination 252 19021 SNAT all -- * * 10.75.22.0/24 0.0.0.0/0 to:24.235.240.15 0 0 SNAT all -- * * 192.168.66.0/24 0.0.0.0/0 to:24.235.240.15
Quoted 8 lines
Chain
ppp0_masq(1 references)pkts bytes target prot opt in out source destination 0 0 SNAT all -- * * 10.75.22.0/24 0.0.0.0/0 to:66.11.173.224 0 0 SNAT all -- * * 192.168.66.0/24 0.0.0.0/0 to:66.11.173.224Is this a problem in that iproute2 selects the default route before SNATting is done to change the source address of the packet, which does not happen of course until POSTROUTING?
The answer to this is of course no. Because the source address of the erroneously-sent-via-eth1 packet (#4 in the above trace) has been correctly re-written (NATted) to 66.11.173.224, then according to the SNAT rules above, the packet is being sent through the correct interface (ppp0).
What still remains inconsistent however is that according to the rules above and currently on my gateway, the "ppp0_masq" rules show 0 hits. How can the source address be correctly re-written to 66.11.173.224 and the rule that does the re-writing show 0 hits?
There is definitely something fishy going on here.
b.
Re: [LARTC] simple dual Internet connection setup not sending
In reply to Brian J. MurrellQuoted 1 line · Brian J. Murrell
To followup on my own posting, with more information...
And yet more...
Quoted 10 lines · Brian J. Murrell
Quoted 4 lines
I have a very simple setup exactly as described in the HOWTO section " 4.2. Routing for multiple uplinks/providers".
One is cable (eth1: dhcp) and the other is PPPoE (ppp0).
These are both on the same physical interface, eth1. IOW, the PPPoE packets are sent to the PPPoE "modem" on eth1. eth1 is also plugged into the cable provider's "modem" as such:
Quoted 7 lines
+---------- Cable Modem
+--------+ |
| | +--+--+
| GW eth1 ------| HUB |
| | +--+--+
+--------+ |
+---------- PPPoE ModemWhich is irrelevant. I have just put a third NIC in the machine to put the PPPoE and Cable connections on different NICs and still the same problem. Packets have PPPoE's source address, but are sent physically on Cable connected NIC.
b.
Re: [LARTC] simple dual Internet connection setup not sending
In reply to Brian J. MurrellQuoted 7 lines · diab
iirc, to have two working internet connections on one (nat'ing) computer you basically need two things (in my example its eth0 and eth1)
1) SNAT to the right source address, like
iptables -A POSTROUTING -j nat -t SNAT [-s from.where or -d to.where]\
--to-source source.addr.of.eth0
Surely you mean -t nat -j SNAT?
Quoted 2 lines
iptables -A POSTROUTING -j nat -t SNAT [-s from.where or -d to.where]\
--to-source source.addr.of.eth1
Ditto on the transposition of -j and -t.
But these two iptables rules conflict with each other. If -s "from.where" is my internal lan and the same in both rules, they are both trying to do the SNATting of the same packets. In my two rules, I added a -o <iface> (where <iface> is the interface matching the source.addr.of.<iface>).
Quoted 3 lines
2) two routing tables, like
ip route add default via eth0.gateway.ip.address dev eth0 table 1
got it:
ip route add 0/0 via 66.11.190.1 dev ppp0 table 1
Quoted 1 line
ip route add default via eth1.gateway.ip.address dev eth1 table 2
got it:
ip route add 0/0 via 24.235.240.1 dev eth1 table 2
Quoted 10 lines
maybe you dont even need the "via xx" thing, the dev xxx is enough.
then you can classify packets to use the connection you want using
ip rule add WHATEVER lookup N (whatever could be "to x.x.x.x" or "from x.x.x.x", same as in the SNAT example, N could be 1 or 2)if you want the router to respond to packets correcty (ie. to answer ping on both interfaces) you need to
ip rule add iif eth0 lookup 1
ip rule add iif eth1 lookup 2
I have:
ip rule add from 66.11.173.224 lookup 1
ip rule add from 24.235.240.15 lookup 2
what is "iif" in your above examples? I don't see an "iif" syntax when I do "ip rule help". I get:
Usage: ip rule [ list | add | del ] SELECTOR ACTION
SELECTOR := [ from PREFIX ] [ to PREFIX ] [ tos TOS ] [ fwmark FWMARK ]
[ dev STRING ] [ pref NUMBER ]
ACTION := [ table TABLE_ID ] [ nat ADDRESS ]
[ prohibit | reject | unreachable ]
[ realms [SRCREALM/]DSTREALM ]TABLE_ID := [ local | main | default | NUMBER ]
Thanx much for your input!
b.
Re: [LARTC] simple dual Internet connection setup not sending
In reply to Brian J. MurrellQuoted 4 lines · diab
yes they are conflicting with each other.. i thought that you could select which connection the packets should be using either based on the address the packets are coming FROM (-s some.ip.on.the.lan) or going TO (-d wan.destination.address.).
No. The problem is that outbound reply packets (i.e. a SYN-ACK packet) to incoming packets (i.e. SYN) are being NATted correctly (i.e. they have the correct source address) they are just not being put on the right interface. They are being put on the interface of the default route in the main routing table.
Quoted 6 lines
iif is the interface packets are coming in (there is also oif).. if it's not a static ip address it might be convenient not having to use the IP of the connection but the interface. (same goes for the "via XX when you are doing "ip route add default dev XY table N")
if you do "man ip" it reads (ip rule add/ip rule del):
~sigh~ My man page for "ip" says only:
NAME
ip - TCP/IP interface configuration and routing utility
SYNTAX
ip
DESCRIPTION
This utility allows you to configure your network interfaces in various ways.
OPTIONS
For the complete command reference please look at the following docu- ment:
/usr/share/doc/iproute-2.4.7/ip-cref.ps
SEE ALSO
ifconfig(8), route(8), netstat(8), arp(8), rarp(8), ipchains(8)
AUTHORS
Alexey Kuznetsov <[email protected]>
and no "/usr/share/doc/iproute-2.4.7/ip-cref.ps" exists.
Quoted 6 lines
iif NAME
select the incoming device to match. If the interface is loopback, the rule only matches packets originating from this host. This means that you may create separate routing tables for forwarded and local packets and, hence, com
pletely segregate them.
OK. But I don't know the device to use. That is the *whole point* of the ip rule add (from <iface address> lookup <table>) isn't it? To select the routing table (and therefore the outbound device) to send the return packets on.
Maybe I am completely missing something in your explanation.
b.
Re: [LARTC] simple dual Internet connection setup not sendingreturn
In reply to Brian J. Murrell"Brian J. Murrell" wrote:
I could be way off base here, but I (obviously) don't think so!
Quoted 11 lines · gypsy
Quoted 2 lines
Guessing from the lack of any mention of KeepState
KeepState? If you are referring to:
52459 2774K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
rules, I have those sprinkled throughout my ruleset where necessary. The iptables "snippet" I included in my previous message was just that. Just the relevant portion that does the NATting.
No, I am referring to http://www.ssi.bg/~ja/nano.txt which is a MUST READ for you, like it or not.
Quoted 7 lines
Quoted 2 lines
in your iptables
setup,Like I said, the RELATED,ESTABLISHED state rules are in there. My full set of iptables rules is >400. I did not see see a need to post that fully here.
Quoted 5 lines
Quoted 2 lines
my guess is that you ignored the advice to vist Julian
Anastasov's web site.No I didn't ignore it. But what that site is promoting is some kind of floppy disk based router distribution or something.
There is a lot of stuff on Julian's site, but I see nothing referring to a floppy disk based router. Use the nano.txt URL above.
Quoted 7 lines
Quoted 2 lines
Start with this:
http://www.geocities.com/mctiew/ffw/dual.htmI am not looking to replace/rebuild my whole firewall. I simply want to add a second link to my existing one and have the packets use the correct interface -- to travel back out the interface from which they came.
That is not why I sent you there...
Quoted 5 lines
I don't want to do load balancing or failover or anything fancy. I want two interfaces where I use one for all outgoing traffic and the only time the alternate is used is to send response packets to connections that come _in_ that interface or for routes that are specifically directed through that interface via a routing table entry.
Yes, you DO want something fancy.
Quoted 9 lines
Quoted 3 lines
You should also google LARTC "Finally: A working case of two adsl load balance". Read Ron Senykoff's post "load balance a file download across two connections - success!".
Interesting. Followed a few links too. Looks like a lot of bells and whistles I am not really looking for (load balancing and failover, etc.) but there is some hint of indication that there is a patch needed to make sure NAT uses the right physical interface. Maybe I will go bug the netfilter guys to see if this is the case.
Perhaps you are not looking for bells and whistles, but you certainly need correct routing tables. Netfilter has nothing to do with what you need except that which is contained in nano.txt.
Quoted 2 lines
Thanx,
b.
gypsy
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/