Wierd Policy Routing Behaviour
Hi there
I'm trying to set up policy routing in a simple scenario without further success. My linux router is connected to two different cable ISPs. One of these links (hereafter named A) permits traffic to SMTP servers while the other (link B) doesn't.
What I am trying to set up is: all traffic should flow through link B and only SMTP traffic through link A.
What I am doing:
Marking all packets from my intranet with source port within the range 1024 to 65535 and with destination port 25 with mark 0x3. I am also marking packets from our internal SMTP server with source port 25 to any other port with the same mark. Marks are being made in the OUTPUT chain (using iptables) as follows:
iptables -t mangle -A OUTPUT -p tcp --sport 1024:65535 --dport 25 -j MARK --set-mark 3
iptables -t mangle -A OUTPUT -p tcp -s <MY SMTP SERVER> --sport 25 -j MARK --set-mark 3
Then, I have created a new routing table named SMTP-ROUTE with default gateway being the default gateway for link A as follows:
ip ro add default via <A's default GW> table SMTP-ROUTE
To complete the setup, I've added a rule stating that all packets marked with 0x3 should use SMTP-ROUTE with this command:
ip ru add fwmark 3 table SMTP-ROUTE
Yes, I've flushed the routing cache with:
ip ro flush cache
Default route in main routing table is B's gateway.
I don't know why, but SMTP traffic keeps going through B's gateway, instead of A's, as expected.
I've sniffed the network and, in fact, the packets are trying to go out through B's gateway.
Can anyone please point what I am doing wrong?
Thank you in advance
Signature · 10 lines
Sellaro Agente Livre - Linux Community (www.agentelivre.org) PGP Key ID: 3ADF8645 PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553 2242 A1D0 17C0 3ADF 8645 _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Sellaro,
I'm not sure about this, (others on the list may confirm), but I think you want to use --set-tos instead of --set-mark.
My understanding is that the mark only exists while the packet is on the local machine, but when you set the ToS flags, then you are changing the packet.
Best of luck,
-Martin
: Hi there
:
: I'm trying to set up policy routing in a simple scenario without further : success. My linux router is connected to two different cable ISPs. One : of these links (hereafter named A) permits traffic to SMTP servers while : the other (link B) doesn't.
:
: What I am trying to set up is: all traffic should flow through link B : and only SMTP traffic through link A.
:
: What I am doing:
:
: Marking all packets from my intranet with source port within the range : 1024 to 65535 and with destination port 25 with mark 0x3. I am also : marking packets from our internal SMTP server with source port 25 to any : other port with the same mark. Marks are being made in the OUTPUT chain : (using iptables) as follows:
:
: iptables -t mangle -A OUTPUT -p tcp --sport 1024:65535 --dport 25 -j : MARK --set-mark 3
:
: iptables -t mangle -A OUTPUT -p tcp -s <MY SMTP SERVER> --sport 25 -j : MARK --set-mark 3
:
: Then, I have created a new routing table named SMTP-ROUTE with default : gateway being the default gateway for link A as follows:
:
: ip ro add default via <A's default GW> table SMTP-ROUTE
:
: To complete the setup, I've added a rule stating that all packets marked : with 0x3 should use SMTP-ROUTE with this command:
:
: ip ru add fwmark 3 table SMTP-ROUTE
:
: Yes, I've flushed the routing cache with:
:
: ip ro flush cache
:
: Default route in main routing table is B's gateway.
:
: I don't know why, but SMTP traffic keeps going through B's gateway, : instead of A's, as expected.
:
: I've sniffed the network and, in fact, the packets are trying to go out : through B's gateway.
:
: Can anyone please point what I am doing wrong?
:
: Thank you in advance
:
Signature · 6 lines
Martin A. Brown --- SecurePipe, Inc. --- [email protected] _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 2 lines · Martin A. Brown
I'm not sure about this, (others on the list may confirm), but I think you want to use --set-tos instead of --set-mark.
I have not tried to use ToS as key for classifying the packets. I'll try to, eventhough I think there's nothing wrong with --set-mark.
Quoted 3 lines
My understanding is that the mark only exists while the packet is on the local machine, but when you set the ToS flags, then you are changing the packet.
But the packets are being marked on the router itself. That's why I think there is nothing wrong (explicitlly, at least) with the solution I'm trying to use. Anyway, I'll wait for comments from others on the list.
Thank you for your help.
Signature · 10 lines
Sellaro Agente Livre - Linux Community (www.agentelivre.org) PGP Key ID: 3ADF8645 PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553 2242 A1D0 17C0 3ADF 8645 _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Without iproute
iptables -t nat -A POSTROUTING -s $SMTP -p tcp --sport 1024: --dport 25 -j SNAT --to-source $IP_B
Quoted 43 lines · Sellaro
Hi there
I'm trying to set up policy routing in a simple scenario without further success. My linux router is connected to two different cable ISPs. One of these links (hereafter named A) permits traffic to SMTP servers while the other (link B) doesn't.
What I am trying to set up is: all traffic should flow through link B and only SMTP traffic through link A.
What I am doing:
Marking all packets from my intranet with source port within the range 1024 to 65535 and with destination port 25 with mark
0x3. I am also marking packets from our internal SMTP server with source port 25 to any other port with the same mark. Marks are being made in the OUTPUT chain (using iptables) as follows:iptables -t mangle -A OUTPUT -p tcp --sport 1024:65535 --dport 25 -j MARK --set-mark 3
iptables -t mangle -A OUTPUT -p tcp -s <MY SMTP SERVER> --sport 25 -j MARK --set-mark 3
Then, I have created a new routing table named SMTP-ROUTE with default gateway being the default gateway for link A as follows:
ip ro add default via <A's default GW> table SMTP-ROUTE
To complete the setup, I've added a rule stating that all packets marked with
0x3should use SMTP-ROUTE with this command:ip ru add fwmark 3 table SMTP-ROUTE
Yes, I've flushed the routing cache with:
ip ro flush cache
Default route in main routing table is B's gateway.
I don't know why, but SMTP traffic keeps going through B's gateway, instead of A's, as expected.
Quoted 18 lines
I've sniffed the network and, in fact, the packets are trying to go out through B's gateway.
Can anyone please point what I am doing wrong?
Thank you in advance
Signature · 10 lines
Sellaro Agente Livre - Linux Community (www.agentelivre.org) PGP Key ID: 3ADF8645 PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553 2242 A1D0 17C0 3ADF 8645 _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
-----------------------------------
mailto:[email protected]
BR
Alexey Talikov
FORTEK
-----------------------------------
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 2 lines · Alexey Talikov
Without iproute
iptables -t nat -A POSTROUTING -s $SMTP -p tcp --sport 1024: --dport 25 -j SNAT --to-source $IP_B
Source NAT is not what I want to do. I really want to send SMTP packets through a different route.
Signature · 10 lines
Sellaro Agente Livre - Linux Community (www.agentelivre.org) PGP Key ID: 3ADF8645 PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553 2242 A1D0 17C0 3ADF 8645 _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
IP: advanced router (CONFIG_IP_ADVANCED_ROUTER) [Y/n/?]
IP: policy routing (CONFIG_IP_MULTIPLE_TABLES) [Y/n/?]
IP: use netfilter MARK value as routing key (CONFIG_IP_ROUTE_FWMARK) [Y/n/?]
iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 1 echo 201 mail.out >> /etc/iproute2/rt_tables
ip rule add fwmark 1 table mail.out
ip route add default via $IP dev $DEV table mail.out
for more detail see Advanced-Routing HOWTO Chapter 11
Quoted 4 lines · Sellaro
Quoted 2 lines · Alexey Talikov
Without iproute
iptables -t nat -A POSTROUTING -s $SMTP -p tcp --sport 1024: --dport 25 -j SNAT --to-source
$IP_B
Quoted 12 lines
Source NAT is not what I want to do. I really want to send SMTP packets through a different route.
Signature · 6 lines
Sellaro Agente Livre - Linux Community (www.agentelivre.org) PGP Key ID: 3ADF8645 PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553 2242 A1D0 17C0 3ADF 8645
-----------------------------------
mailto:[email protected]
BR
Alexey Talikov
FORTEK
-----------------------------------
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 3 lines
IP: advanced router (
CONFIG_IP_ADVANCED_ROUTER) [Y/n/?]
IP: policy routing (CONFIG_IP_MULTIPLE_TABLES) [Y/n/?]
IP: use netfilter MARK value as routing key (CONFIG_IP_ROUTE_FWMARK)
[Y/n/?]
internet:/usr/src/linux# cat .config | grep ROUTER CONFIG_IP_ADVANCED_ROUTER=y
internet:/usr/src/linux# cat .config | grep MULTIPLE CONFIG_IP_MULTIPLE_TABLES=y
internet:/usr/src/linux# cat .config | grep FWMARK CONFIG_IP_ROUTE_FWMARK=y
Quoted 1 line
iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 1
OK, I'm marking with 3, not 1, but I think this will not cause any problem:
iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 3
Quoted 1 line
echo 201 mail.out >>
/etc/iproute2/rt_tables
OK. But table is named only mail.
Quoted 1 line
ip rule add fwmark 1 table mail.out
ip ru add fwmark 3 table mail
ip ru ls
0:
from all lookup local
32765:
from all fwmark 3 lookup mail
32766:
from all lookup main
32767:
from all lookup default
Quoted 1 line
ip route add default via $IP dev $DEV table mail.out
Done.
You've forgoten to mention flushing route cahce. I did it with:
ip ro flush cache
And nothing happens as expected. That's why I am writting to the list.
Thank you for your help.
--
Sellaro
Network Management for the Masses
--
Prendi GRATIS l'email universale che... risparmia: http://www.email.it/f
Sponsor:
Tutti i bulbi in offerta... da Peraga risparmi il 30%! Clicca qui: http://adv2.email.it/cgi-bin/foclick.cgi?mid=448&d=31-5
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Your don't add $DEV
ip route add default via $IP dev $DEV table mail.out
Quoted 56 lines · Sellaro
Quoted 3 lines
IP: advanced router (
CONFIG_IP_ADVANCED_ROUTER) [Y/n/?]
IP: policy routing (CONFIG_IP_MULTIPLE_TABLES) [Y/n/?]
IP: use netfilter MARK value as routing key (CONFIG_IP_ROUTE_FWMARK)[Y/n/?]
internet:/usr/src/linux# cat .config | grep ROUTER CONFIG_IP_ADVANCED_ROUTER=yinternet:/usr/src/linux# cat .config | grep MULTIPLE CONFIG_IP_MULTIPLE_TABLES=yinternet:/usr/src/linux# cat .config | grep FWMARK CONFIG_IP_ROUTE_FWMARK=yQuoted 1 line
iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 1
OK, I'm marking with 3, not 1, but I think this will not cause any problem:
iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 3
Quoted 1 line
echo 201 mail.out >>
/etc/iproute2/rt_tablesOK. But table is named only mail.
Quoted 1 line
ip rule add fwmark 1 table mail.out
ip ru add fwmark 3 table mail
ip ru ls
0:
from all lookup local
32765:
from all fwmark 3 lookup mail
32766:
from all lookup main
32767:
from all lookup defaultQuoted 1 line
ip route add default via $IP dev $DEV table mail.out
Done.
You've forgoten to mention flushing route cahce. I did it with:
ip ro flush cacheAnd nothing happens as expected. That's why I am writting to the list.
Thank you for your help.
--
SellaroNetwork Management for the Masses
Quoted 7 lines
--
Prendi GRATIS l'email universale che... risparmia: http://www.email.it/fSponsor:
Tutti i bulbi in offerta... da Peraga risparmi il 30%! Clicca qui: http://adv2.email.it/cgi-bin/foclick.cgi?midD8&d1-5
-----------------------------------
mailto:[email protected]
BR
Alexey Talikov
FORTEK
-----------------------------------
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 2 lines · Alexey Talikov
Your don't add $DEV
ip route add default via $IP dev $DEV table mail.out
Done. But this is not the problem. I've found a post (reading Advanced Routing Howto again) from Rusty speaking about MASQ and fwmark problems. The post is available at this address:
http://lists.samba.org/pipermail/netfilter/2000-November/006089.html
OK. Now we have a little change on our scenario. All hosts from my INTRANET are able to connect to external SMTP servers properly. However routing itself is not. Sniffing the netowrk shows that from the router the packets are going out through the wrong interface.
I've also added the same same marking rule to OUTPUT table, but it didn't work. Any clues?
Thank you in advance.
Signature · 10 lines
Sellaro Agente Livre - Linux Community (www.agentelivre.org) PGP Key ID: 3ADF8645 PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553 2242 A1D0 17C0 3ADF 8645 _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Version of kernel iptables and iproute , any patches ???
Quoted 15 lines · Sellaro
Quoted 2 lines · Alexey Talikov
Your don't add $DEV
ip route add default via $IP dev $DEV table mail.outDone. But this is not the problem. I've found a post (reading Advanced Routing Howto again) from Rusty speaking about MASQ and fwmark problems. The post is available at this address:
http://lists.samba.org/pipermail/netfilter/2000-November/006089.htmlOK. Now we have a little change on our scenario. All hosts from my INTRANET are able to connect to external SMTP servers properly. However routing itself is not. Sniffing the netowrk shows that from the router the packets are going out through the wrong interface.
Quoted 12 lines
I've also added the same same marking rule to OUTPUT table, but it didn't work. Any clues?
Thank you in advance.
Signature · 6 lines
Sellaro Agente Livre - Linux Community (www.agentelivre.org) PGP Key ID: 3ADF8645 PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553 2242 A1D0 17C0 3ADF 8645
-----------------------------------
mailto:[email protected]
BR
Alexey Talikov
FORTEK
-----------------------------------
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 1 line · Alexey Talikov
Version of kernel iptables and iproute , any patches ???
Sorry for not mentioning before.
kernel: 2.4.18
iproute: 20010824-7
Patches: only HTB 3 into the kernel, of course.
Signature · 10 lines
Sellaro Agente Livre - Linux Community (www.agentelivre.org) PGP Key ID: 3ADF8645 PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553 2242 A1D0 17C0 3ADF 8645 _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
add input interface in rule because this table general for all interfaces also add ip if possible iptables -t mangle -A PREROUTING -i $DEV -p tcp --dport 25 -j MARK --set-mark 1 if don't work
try
iptables -t mangle -A FORWARD -i $DEV -p tcp --dport 25 -j MARK --set-mark 1 (don't work for old iptables and kernel, missing FORWARD and INPUT chain for mangle table)
if you compile netfilter as modules -- all necessary loaded ?
if don't work try to test mark with tc filter
try to use last iproute2-2.4.7-020116 (possible that the ip route can't see your marks and last htb-3.6 (Thanks Devik !)
Send me results !!
(Sorry for my pure English :(( )
Quoted 18 lines · Sellaro
Quoted 1 line · Alexey Talikov
Version of kernel iptables and iproute , any patches ???
Sorry for not mentioning before.
kernel: 2.4.18
iproute: 20010824-7
Patches: only HTB 3 into the kernel, of course.Signature · 6 lines
Sellaro Agente Livre - Linux Community (www.agentelivre.org) PGP Key ID: 3ADF8645 PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553 2242 A1D0 17C0 3ADF 8645
-----------------------------------
mailto:[email protected]
BR
Alexey Talikov
FORTEK
-----------------------------------
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/