Policy routing and strange packets traversing.
Hi,
Please suppose following config:
Two external interfaces for two different providers On each of them configured NAT for specific IP addr. ie.
4: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb qlen 100
inet 1.1.1.30/30 brd 1.1.1.31 scope global eth2
6: eth4: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb qlen 100
inet 2.2.2.66/27 brd 2.2.2.95 scope global eth4On eth2 NAT outgoing traffic to 1.1.1.30
On eth4 NAT outgoing traffic to 2.2.2.66
Rules related for this interfaces and traffic:
# To be sure that traffic goes to proper gateway
22: from 1.1.1.30 lookup 1
22: from 2.2.2.66 lookup 2
...
# This rules are unnecessary I think but used for diagnostics gateways #by me.
30: from all to 1.1.1.29 lookup 1
30: from all to 2.2.2.65 lookup 2
#Balance tables distributes traffic from LAN.
70: from all iif eth1 lookup balance
# ip r l ta 1
default via 1.1.1.29 dev eth2
# ip r l ta 2
default via 2.2.2.65 dev eth4
# ip r l ta balance
default
nexthop via 1.1.1.29 dev eth2 weight 2
nexthop via 2.2.2.65 dev eth4 weight 3So. Everything works but I have observed some behaviour what I can't understand..
What I expected was that trafic nated to 1.1.1.30 goes throught eth2 and traffic nated to 2.2.2.66 goes throught eth4.
Unfortunatelly when become listening on eth4 with following command:
tcpdump -n -i eth4 src 1.1.1.30
I can see trafiic which I am not expecting on this interface:
1.1.1.30.3145 > 217.98.144.187.20: P 1608:2144(536) ack 1 win 16616 (DF) 1.1.1.30.4282 > 212.77.100.17.5555: . ack 1889 win 17520 (DF)
The simmilar is on eth2:
tcpdump -n -i eth2 src 2.2.2.66
2.2.2.66.6114 > 217.17.41.85.8074: P 58257:58281(24) ack 530714947 win 7506 (DF)
Of course more packets have correct sources [1.1.1.30 for eth2 and 2.2.2.66 on eth4] but I cant see the reason there are some missed packets...
I did experiment and attached iptables DROP rule on POSTROUTING on eth2 and eth4 interfaces to catch bad sourced packets but they didnt catch anything what says for me this "bad" traffic didnt really go through incorrect interfaces.
So that I am confused on this packet traversing.. Could someone explain this behaviour ? Is it OK or I have missed something ?
Regards,
tw
--
-----------
ck.eter.tym.pl
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hello,
Quoted 1 line · Tomasz Wrona
Rules related for this interfaces and traffic:
This looks a bit strange, it is not needed:
Quoted 3 lines
# To be sure that traffic goes to proper gateway
22: from 1.1.1.30 lookup 1
22: from 2.2.2.66 lookup 2
You already have link routes to these IPs in table main
Quoted 3 lines
...
# This rules are unnecessary I think but used for diagnostics gateways #by me.
Yes, you don't need them:
Quoted 4 lines
30: from all to 1.1.1.29 lookup 1
30: from all to 2.2.2.65 lookup 2#Balance tables distributes traffic from LAN.
Don't expect from Netfilter to use correctly the routing,
you have to avoid using "iif" when playing with Netfilter. Just use "from XXX".
Quoted 12 lines
70: from all iif eth1 lookup balance
# ip r l ta 1 default via 1.1.1.29 dev eth2 # ip r l ta 2 default via 2.2.2.65 dev eth4 # ip r l ta balance default nexthop via 1.1.1.29 dev eth2 weight 2 nexthop via 2.2.2.65 dev eth4 weight 3
Quoted 2 lines
So. Everything works but I have observed some behaviour what I can't understand..
I don't know what works but in theory it should not work,
you don't have routes that restrict each ISP traffic through its gateway. May be in your case each of the ISPs allow spoofing.
Quoted 2 lines
What I expected was that trafic nated to 1.1.1.30 goes throught eth2 and traffic nated to 2.2.2.66 goes throught eth4.
Then specify it to be so:
ip rule add prio 20 from 1.1.1.30/30 table 1
ip rule add prio 20 from 2.2.2.66/27 table 2
but you will need rules "from all to all" for
proper default route selelection and source IP autoselection for the masquerading. The normal kernel can not give you this, you need other solutions, eg:
dgd-usage.txt contains example for rules and routes you can use.
Quoted 10 lines
Unfortunatelly when become listening on eth4 with following command:
tcpdump -n -i eth4 src 1.1.1.30
I can see trafiic which I am not expecting on this interface:
1.1.1.30.3145 > 217.98.144.187.20: P 1608:2144(536) ack 1 win 16616 (DF) 1.1.1.30.4282 > 212.77.100.17.5555: . ack 1889 win 17520 (DF)The simmilar is on eth2:
tcpdump -n -i eth2 src 2.2.2.66
2.2.2.66.6114 > 217.17.41.85.8074: P 58257:58281(24) ack 530714947 win 7506 (DF)
Quoted 7 lines
Of course more packets have correct sources [1.1.1.30 for eth2 and 2.2.2.66 on eth4] but I cant see the reason there are some missed packets...
I did experiment and attached iptables DROP rule on POSTROUTING on eth2 and eth4 interfaces to catch bad sourced packets but they didnt catch anything what says for me this "bad" traffic didnt really go through incorrect interfaces.
May be it is the POST_ROUTING who is guilty for selecting
wrong nexthop and you can not notice it, this mistake is visible on device output.
Quoted 2 lines
So that I am confused on this packet traversing.. Could someone explain this behaviour ? Is it OK or I have missed something ?
You can read about such issues, use the above URL
Quoted 6 lines
Regards,
tw
-------------
ck.eter.tym.pl
Regards
--
Julian Anastasov <[email protected]>
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 5 lines · Julian Anastasov
This looks a bit strange, it is not needed:
Quoted 3 lines
# To be sure that traffic goes to proper gateway
22: from 1.1.1.30 lookup 1
22: from 2.2.2.66 lookup 2
Why, It's the same what You pointed me below... ?
Quoted 5 lines
You already have link routes to these IPs in table main Yes, you don't need them:
Quoted 2 lines
30: from all to 1.1.1.29 lookup 1
30: from all to 2.2.2.65 lookup 2
OK, but I process main table after all manual typed rules... but never mind its not issue ;)
Quoted 5 lines
Quoted 1 line
#Balance tables distributes traffic from LAN.
Don't expect from Netfilter to use correctly the routing,
you have to avoid using "iif" when playing with Netfilter. Just use "from XXX".
Hmmm... I cant understand what has netfilter to do with "iif" parameter ? What I want to achieve is to catch all incoming traffic on eth1..
Quoted 12 lines
Quoted 12 lines
70: from all iif eth1 lookup balance
# ip r l ta 1 default via 1.1.1.29 dev eth2 # ip r l ta 2 default via 2.2.2.65 dev eth4 # ip r l ta balance default nexthop via 1.1.1.29 dev eth2 weight 2 nexthop via 2.2.2.65 dev eth4 weight 3
Quoted 6 lines
Quoted 2 lines
So. Everything works but I have observed some behaviour what I can't understand..
I don't know what works but in theory it should not work,
you don't have routes that restrict each ISP traffic through its gateway. May be in your case each of the ISPs allow spoofing.
Well,look at top of the mail for first rules what You say are wrong..
Quoted 7 lines
Quoted 2 lines
What I expected was that trafic nated to 1.1.1.30 goes throught eth2 and traffic nated to 2.2.2.66 goes throught eth4.
Then specify it to be so:
ip rule add prio 20 from 1.1.1.30/30 table 1
ip rule add prio 20 from 2.2.2.66/27 table 2
This is exact the same what firs rules on top of mail, Am I really wrong ?
Quoted 4 lines
but you will need rules "from all to all" for
proper default route selelection and source IP autoselection for the masquerading.
Balance table catches all traffic from LAN to inet.Thats all what I need.
Quoted 6 lines
The normal kernel can not give you this, you
need other solutions, eg:dgd-usage.txt contains example for rules and routes you can use.
Hmm... Maybe I am wrong but It's related to NAT multiple gateways on single interface not on different what I have...
There shouldn't be problem what I read in this article.
Quoted 10 lines
Quoted 1 line
Unfortunatelly when become listening on eth4 with following command:
May be it is the
POST_ROUTINGwho is guilty for selecting
wrong nexthop and you can not notice it, this mistake is visible on device output.Quoted 2 lines
So that I am confused on this packet traversing.. Could someone explain this behaviour ? Is it OK or I have missed something ?
You can read about such issues, use the above URL
I will dig it still.
Thank You for support
tw
Signature · 9 lines
---------------- ck.eter.tym.pl "Never let shooling disturb Your education" _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hello,
Quoted 7 lines · Tomasz Wrona
Quoted 5 lines
This looks a bit strange, it is not needed:
Quoted 3 lines
# To be sure that traffic goes to proper gateway
22: from 1.1.1.30 lookup 1
22: from 2.2.2.66 lookup 2Why, It's the same what You pointed me below... ?
OK, I overlooked it
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
30: from all to 1.1.1.29 lookup 1
30: from all to 2.2.2.65 lookup 2OK, but I process main table after all manual typed rules... but never mind its not issue ;)
It is not good to put table main after other rules, it can
be used only to override route in table main. For example, why traffic from 1.1.1.29 to some internal IP should go to the ISP gateway (table 1)?
Quoted 6 lines
Quoted 3 lines
Don't expect from Netfilter to use correctly the routing,
you have to avoid using "iif" when playing with Netfilter. Just use "from XXX".Hmmm... I cant understand what has netfilter to do with "iif" parameter ? What I want to achieve is to catch all incoming traffic on eth1..
There are some places that can use output rerouting where
the iif parameter is ignored. And second, the normal kernel relies on the routing cache to keep persistence for each NAT connection to its selected nexthop. There is no guarantee that it will work for the whole connection life.
Quoted 5 lines
Quoted 4 lines
but you will need rules "from all to all" for
proper default route selelection and source IP autoselection for the masquerading.Balance table catches all traffic from LAN to inet.Thats all what I need.
It does not work all the time.
Quoted 6 lines
Quoted 3 lines
dgd-usage.txt contains example for rules and routes you can use.
Hmm... Maybe I am wrong but It's related to NAT multiple gateways on single interface not on different what I have...
Not exactly true, it is related to making sure each NAT
conn is bound to its allowed path(s), no matter how many interfaces are used. Selecting different nexthop should be allowed only if it is alternative allowed from the routing rules.
Quoted 1 line
There shouldn't be problem what I read in this article.
Then why you see traffic to the wrong gateway?
Regards
--
Julian Anastasov <[email protected]>
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/