Skip to contentSkip to conversation

Policy routing and strange packets traversing.

LARTC Tomasz Wrona 2 participants 4 messages Mar 1, 2003 – Mar 2, 2003

lore
Share & tools
lore
#0 lore raw

Hi,

Please suppose following config:

Two external interfaces for two different providers On each of them configured NAT for specific IP addr. ie.

4: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb qlen 100

    inet 1.1.1.30/30 brd 1.1.1.31 scope global eth2
6: eth4: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb qlen 100
    inet 2.2.2.66/27 brd 2.2.2.95 scope global eth4

On eth2 NAT outgoing traffic to 1.1.1.30
On eth4 NAT outgoing traffic to 2.2.2.66

Rules related for this interfaces and traffic:

# To be sure that traffic goes to proper gateway
22: from 1.1.1.30 lookup 1
22: from 2.2.2.66 lookup 2
...
# This rules are unnecessary I think but used for diagnostics gateways #by me.
30: from all to 1.1.1.29 lookup 1
30: from all to 2.2.2.65 lookup 2

#Balance tables distributes traffic from LAN.
70: from all iif eth1 lookup balance

# ip r l ta 1
default via 1.1.1.29 dev eth2
# ip r l ta 2
default via 2.2.2.65 dev eth4
# ip r l ta balance
default
        nexthop via 1.1.1.29  dev eth2 weight 2
        nexthop via 2.2.2.65  dev eth4 weight 3

So. Everything works but I have observed some behaviour what I can't understand..

What I expected was that trafic nated to 1.1.1.30 goes throught eth2 and traffic nated to 2.2.2.66 goes throught eth4.

Unfortunatelly when become listening on eth4 with following command:
tcpdump -n -i eth4 src 1.1.1.30
I can see trafiic which I am not expecting on this interface:
1.1.1.30.3145 > 217.98.144.187.20: P 1608:2144(536) ack 1 win 16616 (DF) 1.1.1.30.4282 > 212.77.100.17.5555: . ack 1889 win 17520 (DF)

The simmilar is on eth2:
tcpdump -n -i eth2 src 2.2.2.66
2.2.2.66.6114 > 217.17.41.85.8074: P 58257:58281(24) ack 530714947 win 7506 (DF)

Of course more packets have correct sources [1.1.1.30 for eth2 and 2.2.2.66 on eth4] but I cant see the reason there are some missed packets...
I did experiment and attached iptables DROP rule on POSTROUTING on eth2 and eth4 interfaces to catch bad sourced packets but they didnt catch anything what says for me this "bad" traffic didnt really go through incorrect interfaces.

So that I am confused on this packet traversing.. Could someone explain this behaviour ? Is it OK or I have missed something ?

Regards,
tw
--

-----------
ck.eter.tym.pl

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#1 lore raw

Hello,

Quoted 1 line · Tomasz Wrona

Rules related for this interfaces and traffic:

This looks a bit strange, it is not needed:

Quoted 3 lines

# To be sure that traffic goes to proper gateway
22: from 1.1.1.30 lookup 1
22: from 2.2.2.66 lookup 2

You already have link routes to these IPs in table main

Quoted 3 lines

...
# This rules are unnecessary I think but used for diagnostics gateways #by me.

Yes, you don't need them:

Quoted 4 lines

30: from all to 1.1.1.29 lookup 1
30: from all to 2.2.2.65 lookup 2

#Balance tables distributes traffic from LAN.

Don't expect from Netfilter to use correctly the routing,
you have to avoid using "iif" when playing with Netfilter. Just use "from XXX".

Quoted 12 lines

70: from all iif eth1 lookup balance

# ip r l ta 1
default via 1.1.1.29 dev eth2
# ip r l ta 2
default via 2.2.2.65 dev eth4
# ip r l ta balance
default
        nexthop via 1.1.1.29  dev eth2 weight 2
        nexthop via 2.2.2.65  dev eth4 weight 3
Quoted 2 lines

So. Everything works but I have observed some behaviour what I can't understand..

I don't know what works but in theory it should not work,
you don't have routes that restrict each ISP traffic through its gateway. May be in your case each of the ISPs allow spoofing.

Quoted 2 lines

What I expected was that trafic nated to 1.1.1.30 goes throught eth2 and traffic nated to 2.2.2.66 goes throught eth4.

Then specify it to be so:

ip rule add prio 20 from 1.1.1.30/30 table 1
ip rule add prio 20 from 2.2.2.66/27 table 2

but you will need rules "from all to all" for
proper default route selelection and source IP autoselection for the masquerading. The normal kernel can not give you this, you need other solutions, eg:

http://www.ssi.bg/~ja/#routes

dgd-usage.txt contains example for rules and routes you can use.

Quoted 10 lines

Unfortunatelly when become listening on eth4 with following command:
tcpdump -n -i eth4 src 1.1.1.30
I can see trafiic which I am not expecting on this interface:
1.1.1.30.3145 > 217.98.144.187.20: P 1608:2144(536) ack 1 win 16616 (DF) 1.1.1.30.4282 > 212.77.100.17.5555: . ack 1889 win 17520 (DF)

The simmilar is on eth2:
tcpdump -n -i eth2 src 2.2.2.66
2.2.2.66.6114 > 217.17.41.85.8074: P 58257:58281(24) ack 530714947 win 7506 (DF)

Quoted 7 lines

Of course more packets have correct sources [1.1.1.30 for eth2 and 2.2.2.66 on eth4] but I cant see the reason there are some missed packets...
I did experiment and attached iptables DROP rule on POSTROUTING on eth2 and eth4 interfaces to catch bad sourced packets but they didnt catch anything what says for me this "bad" traffic didnt really go through incorrect interfaces.

May be it is the POST_ROUTING who is guilty for selecting
wrong nexthop and you can not notice it, this mistake is visible on device output.

Quoted 2 lines

So that I am confused on this packet traversing.. Could someone explain this behaviour ? Is it OK or I have missed something ?

You can read about such issues, use the above URL

Quoted 6 lines

Regards,
tw
--

-----------
ck.eter.tym.pl

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#2 lore raw
In reply to Tomasz Wrona
Quoted 5 lines · Julian Anastasov

This looks a bit strange, it is not needed:

Quoted 3 lines

# To be sure that traffic goes to proper gateway
22: from 1.1.1.30 lookup 1
22: from 2.2.2.66 lookup 2

Why, It's the same what You pointed me below... ?

Quoted 5 lines

You already have link routes to these IPs in table main Yes, you don't need them:

Quoted 2 lines

30: from all to 1.1.1.29 lookup 1
30: from all to 2.2.2.65 lookup 2

OK, but I process main table after all manual typed rules... but never mind its not issue ;)

Quoted 5 lines
Quoted 1 line

#Balance tables distributes traffic from LAN.

Don't expect from Netfilter to use correctly the routing,
you have to avoid using "iif" when playing with Netfilter. Just use "from XXX".

Hmmm... I cant understand what has netfilter to do with "iif" parameter ? What I want to achieve is to catch all incoming traffic on eth1..

Quoted 12 lines
Quoted 12 lines

70: from all iif eth1 lookup balance

# ip r l ta 1
default via 1.1.1.29 dev eth2
# ip r l ta 2
default via 2.2.2.65 dev eth4
# ip r l ta balance
default
        nexthop via 1.1.1.29  dev eth2 weight 2
        nexthop via 2.2.2.65  dev eth4 weight 3
Quoted 6 lines
Quoted 2 lines

So. Everything works but I have observed some behaviour what I can't understand..

I don't know what works but in theory it should not work,
you don't have routes that restrict each ISP traffic through its gateway. May be in your case each of the ISPs allow spoofing.

Well,look at top of the mail for first rules what You say are wrong..

Quoted 7 lines
Quoted 2 lines

What I expected was that trafic nated to 1.1.1.30 goes throught eth2 and traffic nated to 2.2.2.66 goes throught eth4.

Then specify it to be so:

ip rule add prio 20 from 1.1.1.30/30 table 1
ip rule add prio 20 from 2.2.2.66/27 table 2

This is exact the same what firs rules on top of mail, Am I really wrong ?

Quoted 4 lines

but you will need rules "from all to all" for
proper default route selelection and source IP autoselection for the masquerading.

Balance table catches all traffic from LAN to inet.Thats all what I need.

Quoted 6 lines

The normal kernel can not give you this, you
need other solutions, eg:

http://www.ssi.bg/~ja/#routes

dgd-usage.txt contains example for rules and routes you can use.

Hmm... Maybe I am wrong but It's related to NAT multiple gateways on single interface not on different what I have...
There shouldn't be problem what I read in this article.

Quoted 10 lines
Quoted 1 line

Unfortunatelly when become listening on eth4 with following command:

May be it is the POST_ROUTING who is guilty for selecting
wrong nexthop and you can not notice it, this mistake is visible on device output.

Quoted 2 lines

So that I am confused on this packet traversing.. Could someone explain this behaviour ? Is it OK or I have missed something ?

You can read about such issues, use the above URL

I will dig it still.

Thank You for support
tw

Signature · 9 lines
----------------
 ck.eter.tym.pl

"Never let shooling disturb Your education"


_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#3 lore raw
In reply to Tomasz Wrona

Hello,

Quoted 7 lines · Tomasz Wrona
Quoted 5 lines

This looks a bit strange, it is not needed:

Quoted 3 lines

# To be sure that traffic goes to proper gateway
22: from 1.1.1.30 lookup 1
22: from 2.2.2.66 lookup 2

Why, It's the same what You pointed me below... ?

OK, I overlooked it

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

30: from all to 1.1.1.29 lookup 1
30: from all to 2.2.2.65 lookup 2

OK, but I process main table after all manual typed rules... but never mind its not issue ;)

It is not good to put table main after other rules, it can
be used only to override route in table main. For example, why traffic from 1.1.1.29 to some internal IP should go to the ISP gateway (table 1)?

Quoted 6 lines
Quoted 3 lines

Don't expect from Netfilter to use correctly the routing,
you have to avoid using "iif" when playing with Netfilter. Just use "from XXX".

Hmmm... I cant understand what has netfilter to do with "iif" parameter ? What I want to achieve is to catch all incoming traffic on eth1..

There are some places that can use output rerouting where
the iif parameter is ignored. And second, the normal kernel relies on the routing cache to keep persistence for each NAT connection to its selected nexthop. There is no guarantee that it will work for the whole connection life.

Quoted 5 lines
Quoted 4 lines

but you will need rules "from all to all" for
proper default route selelection and source IP autoselection for the masquerading.

Balance table catches all traffic from LAN to inet.Thats all what I need.

It does not work all the time.

Quoted 6 lines
Quoted 3 lines

http://www.ssi.bg/~ja/#routes

dgd-usage.txt contains example for rules and routes you can use.

Hmm... Maybe I am wrong but It's related to NAT multiple gateways on single interface not on different what I have...

Not exactly true, it is related to making sure each NAT
conn is bound to its allowed path(s), no matter how many interfaces are used. Selecting different nexthop should be allowed only if it is alternative allowed from the routing rules.

Quoted 1 line

There shouldn't be problem what I read in this article.

Then why you see traffic to the wrong gateway?

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/