Masq/route based on port
I have following setup:
- eth0 is an internal network
- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)
I would like to masquerade port 80 through eth2, but all other traffic should be masq'ed through eth1.
My routing configuration:
(default route in main table is 1.1.1.1)
ip rule add fwmark 2 pref 1002 table 666
ip route flush table 666
ip route add default via 2.2.2.1 dev eth3 proto static table 666 ip route flush cache
My firewall configuration:
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
--set-mark 2
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128Unfortunately, this does not work. Outgoing packets are fine. Incoming packets on port 80 are not de-masqueraded and do not reach the internal hosts.
Also, if I change the ip rule above to be based on the source address (instead of a mark), connections start working fine.
Here is the output of 'ip rule ls', to prove that I do have fwmark compiled:
0: from all lookup local
1002: from all fwmark 2 lookup http
32766: from all lookup main
32767: from all lookup 253I am wondering if there is some kind of bug related to the interaction between fwmark and NAT. Any ideas?
Thanks,
Miron Cuperman
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
This doesn't seem right:
Quoted 5 lines
My firewall configuration:
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
--set-mark 2
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128 iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128
I'm not the expert, but I don't think you want to SNAT.
You're marking inbound packets with destination port 80 and then putting rules in the POSTROUTING table to fudge in a different IP address for outbound packets. That doesn't seem right. It doesn't redirect inbound packets the way you want.
I think you want to DNAT instead of SNAT and forget about marking packets. Set up some PREROUTING rules and DNAT all incoming port 80 stuff over to the interface you want. That should be all you need to do because the connection tracking should take care of getting the reply packets from your internal web server back to where they belong.
Verify this with the experts before you do it, but I think I'm right on this one.
- Greg Scott
-----Original Message-----
From: Miron [mailto:[email protected]]
Sent: Thursday, December 06, 2001 3:58 AM
To: [email protected]
Subject: [LARTC] Masq/route based on port
I have following setup:
- eth0 is an internal network
- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)
I would like to masquerade port 80 through eth2, but all other traffic should be masq'ed through eth1.
My routing configuration:
(default route in main table is 1.1.1.1)
ip rule add fwmark 2 pref 1002 table 666
ip route flush table 666
ip route add default via 2.2.2.1 dev eth3 proto static table 666 ip route flush cache
My firewall configuration:
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
--set-mark 2
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128Unfortunately, this does not work. Outgoing packets are fine. Incoming packets on port 80 are not de-masqueraded and do not reach the internal hosts.
Also, if I change the ip rule above to be based on the source address (instead of a mark), connections start working fine.
Here is the output of 'ip rule ls', to prove that I do have fwmark compiled:
0: from all lookup local
1002: from all fwmark 2 lookup http
32766: from all lookup main
32767: from all lookup 253I am wondering if there is some kind of bug related to the interaction between fwmark and NAT. Any ideas?
Thanks,
Miron Cuperman
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://ds9a.nl/2.4Routing/
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
This is a home setup, not a server setup. We have no servers on our network. The reason we want port 80 on eth2 is because eth2 has more download bandwidth. For other protocols we want eth1, because it has more symmetric bandwidth.
Quoted 82 lines · Greg Scott
This doesn't seem right:
Quoted 6 lines
My firewall configuration:
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
--set-mark 2
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128 iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128I'm not the expert, but I don't think you want to SNAT.
You're marking inbound packets with destination port 80 and then putting rules in the POSTROUTING table to fudge in a different IP address for outbound packets. That doesn't seem right. It doesn't redirect inbound packets the way you want.
I think you want to DNAT instead of SNAT and forget about marking packets. Set up some PREROUTING rules and DNAT all incoming port 80 stuff over to the interface you want. That should be all you need to do because the connection tracking should take care of getting the reply packets from your internal web server back to where they belong.
Verify this with the experts before you do it, but I think I'm right on this one.
- Greg Scott
-----Original Message-----
From: Miron [mailto:[email protected]]
Sent: Thursday, December 06, 2001 3:58 AM
To: [email protected]
Subject: [LARTC] Masq/route based on portI have following setup:
- eth0 is an internal network
- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)I would like to masquerade port 80 through eth2, but all other traffic should be masq'ed through eth1.
My routing configuration:
(default route in main table is 1.1.1.1)
ip rule add fwmark 2 pref 1002 table 666
ip route flush table 666
ip route add default via 2.2.2.1 dev eth3 proto static table 666 ip route flush cacheMy firewall configuration:
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK--set-mark 2
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128 iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128Unfortunately, this does not work. Outgoing packets are fine. Incoming packets on port 80 are not de-masqueraded and do not reach the internal hosts.
Also, if I change the ip rule above to be based on the source address (instead of a mark), connections start working fine.
Here is the output of 'ip rule ls', to prove that I do have fwmark compiled:
0: from all lookup local 1002: from all fwmark 2 lookup http 32766: from all lookup main 32767: from all lookup 253I am wondering if there is some kind of bug related to the interaction between fwmark and NAT. Any ideas?
Thanks,
Miron Cuperman
Quoted 9 lines
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://ds9a.nl/2.4Routing/_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
Quoted 4 lines
This is a home setup, not a server setup. We have no servers on our network. The reason we want port 80 on eth2 is because eth2 has more download bandwidth. For other protocols we want eth1, because it has more symmetric bandwidth.
So anything that comes in from the Internet for port 80, no matter the source, you want the reply to go back out on ETH2. And anything that comes in from the Internet other than port 80, you want those replies to go out ETH1. So the web server process is inside your Linux box?
Did I get that much right? Or do I have it backwards?
The Linux box is your internal LAN's default gateway and you want this box to decide which Internet interface to use, based on the destination port your internal client PCs choose?
Hadn't thought about it that way before.
- Greg
-----Original Message-----
From: Miron [mailto:[email protected]]
Sent: Thursday, December 06, 2001 1:03 PM
To: Greg Scott
Cc: [email protected]
Subject: Re: [LARTC] Masq/route based on port
Quoted 25 lines
I have following setup:
- eth0 is an internal network
- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)I would like to masquerade port 80 through eth2, but all other traffic should be masq'ed through eth1.
My routing configuration:
(default route in main table is 1.1.1.1)
ip rule add fwmark 2 pref 1002 table 666
ip route flush table 666
ip route add default via 2.2.2.1 dev eth3 proto static table 666 ip route flush cacheMy firewall configuration:
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK--set-mark 2
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128 iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128
Quoted 8 lines
Unfortunately, this does not work. Outgoing packets are fine. Incoming packets on port 80 are not de-masqueraded and do not reach the internal hosts.
Also, if I change the ip rule above to be based on the source address (instead of a mark), connections start working fine.
Here is the output of 'ip rule ls', to prove that I do have fwmark
compiled:
Quoted 10 lines
0: from all lookup local 1002: from all fwmark 2 lookup http 32766: from all lookup main 32767: from all lookup 253I am wondering if there is some kind of bug related to the interaction between fwmark and NAT. Any ideas?
Thanks,
Miron Cuperman
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
I think we are on the same wavelength, except maybe exactly backwards. ;)
I'm trying to provide fast web browsing for clients that are on the internal network (talking to servers on the Internet). Anything from the Internet with source port 80 should go through eth2. Anything else should go through eth1.
I did some more investigation, and it seems that the Linux box is dropping all incoming packets arriving into eth2. I can see packets with tcpdump, but the applications and NAT don't actually get it. It must be getting dropped in the kernel. It seems that this has something to do with the default route pointing to eth1. I don't understand why the fact that the default route is not pointing to an interface should cause *incoming* packets to be dropped on that interface.
Sorry if my post was confusing.
Quoted 78 lines · Greg Scott
Quoted 5 lines
This is a home setup, not a server setup. We have no servers on our network. The reason we want port 80 on eth2 is because eth2 has more download bandwidth. For other protocols we want eth1, because it has more symmetric bandwidth.
So anything that comes in from the Internet for port 80, no matter the source, you want the reply to go back out on ETH2. And anything that comes in from the Internet other than port 80, you want those replies to go out ETH1. So the web server process is inside your Linux box?
Did I get that much right? Or do I have it backwards?
The Linux box is your internal LAN's default gateway and you want this box to decide which Internet interface to use, based on the destination port your internal client PCs choose?
Hadn't thought about it that way before.
- Greg
-----Original Message-----
From: Miron [mailto:[email protected]]
Sent: Thursday, December 06, 2001 1:03 PM
To: Greg Scott
Cc: [email protected]
Subject: Re: [LARTC] Masq/route based on portQuoted 25 lines
I have following setup:
- eth0 is an internal network
- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)I would like to masquerade port 80 through eth2, but all other traffic should be masq'ed through eth1.
My routing configuration:
(default route in main table is 1.1.1.1)
ip rule add fwmark 2 pref 1002 table 666
ip route flush table 666
ip route add default via 2.2.2.1 dev eth3 proto static table 666 ip route flush cacheMy firewall configuration:
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
--set-mark 2
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128 iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128Quoted 9 lines
Unfortunately, this does not work. Outgoing packets are fine. Incoming packets on port 80 are not de-masqueraded and do not reach the internal hosts.
Also, if I change the ip rule above to be based on the source address (instead of a mark), connections start working fine.
Here is the output of 'ip rule ls', to prove that I do have fwmark
compiled:
Quoted 8 lines
0: from all lookup local 1002: from all fwmark 2 lookup http 32766: from all lookup main 32767: from all lookup 253I am wondering if there is some kind of bug related to the interaction between fwmark and NAT. Any ideas?
Quoted 8 lines
Quoted 3 lines
Thanks,
Miron Cuperman_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
Okay, I figured it out. The secret is to set rp_filter to zero on the non-default interface. I guess the kernel thought that the packets were source-routed because of the fact that the interface received packets it could not route.
That was frustrating, yet fun... ;)
Quoted 16 lines · Miron
I think we are on the same wavelength, except maybe exactly backwards. ;)
I'm trying to provide fast web browsing for clients that are on the internal network (talking to servers on the Internet). Anything from the Internet with source port 80 should go through eth2. Anything else should go through eth1.
I did some more investigation, and it seems that the Linux box is dropping all incoming packets arriving into eth2. I can see packets with tcpdump, but the applications and NAT don't actually get it. It must be getting dropped in the kernel. It seems that this has something to do with the default route pointing to eth1. I don't understand why the fact that the default route is not pointing to an interface should cause *incoming* packets to be dropped on that interface.
Quoted 83 lines
Sorry if my post was confusing.
Quoted 79 lines · Greg Scott
Quoted 5 lines
This is a home setup, not a server setup. We have no servers on our network. The reason we want port 80 on eth2 is because eth2 has more download bandwidth. For other protocols we want eth1, because it has more symmetric bandwidth.
So anything that comes in from the Internet for port 80, no matter the source, you want the reply to go back out on ETH2. And anything that
comes in from the Internet other than port 80, you want those replies to go out ETH1. So the web server process is inside your Linux box? Did I get that much right? Or do I have it backwards?
The Linux box is your internal LAN's default gateway and you want this box to decide which Internet interface to use, based on the destination port your internal client PCs choose?
Hadn't thought about it that way before.- Greg
-----Original Message-----
From: Miron [mailto:[email protected]]
Sent: Thursday, December 06, 2001 1:03 PM
To: Greg Scott
Cc: [email protected]
Subject: Re: [LARTC] Masq/route based on portQuoted 25 lines
I have following setup:
- eth0 is an internal network
- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)I would like to masquerade port 80 through eth2, but all other traffic should be masq'ed through eth1.
My routing configuration:
(default route in main table is 1.1.1.1)
ip rule add fwmark 2 pref 1002 table 666
ip route flush table 666
ip route add default via 2.2.2.1 dev eth3 proto static table 666 ip route flush cacheMy firewall configuration:
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
--set-mark 2
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128 iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128Quoted 9 lines
Unfortunately, this does not work. Outgoing packets are fine. Incoming packets on port 80 are not de-masqueraded and do not reach the internal hosts.
Also, if I change the ip rule above to be based on the source address (instead of a mark), connections start working fine.
Here is the output of 'ip rule ls', to prove that I do have fwmark
compiled:
Quoted 11 lines
0: from all lookup local 1002: from all fwmark 2 lookup http 32766: from all lookup main 32767: from all lookup 253I am wondering if there is some kind of bug related to the interaction between fwmark and NAT. Any ideas?
Thanks,
Miron Cuperman
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/