POSTROUTING SNAT only reply packets
Hi,
I need to modify the reply packets of one web server to allow the connections between a webserver and client using a load balancer.
The client connections goes to a load balancer, the load balancer forwards the connection to a one web server changing the destination ip, the web server anwser the client with it's own ip address without passing again for the load balancer. In order to stablish the connection, the client needs to receive the web server answer with the correct ip address (in this case, the load balancer VIP address), in other case it receives ACK that it doesn't know about it and the connections is not ESTABLISHED.
I've doing some testing and seems that iptables only do SNAT on NEW connections, and I need to change the ip address of replied packets. Anybody know some workaround? If anobody do not know some workaround can you confirm that it's not posible to do this with iptables?
I've tried the next ip tables rules and only work when I do NEW connections from the web server.
-A POSTROUTING -o br0 -s WE_SERVER_ADDR -p tcp -m tcp --sport 80 --dport 1024:65535 -j SNAT --to-source LOAD_BALANCER_ADDR
Thanks in advance.
Isaac González
Hi,
2011-01-17 11:16 keltezéssel, GMail Isaac Gonzalez írta:
Quoted 4 lines
Hi,
I've doing some testing and seems that iptables only do SNAT on NEW connections, and I need to change the ip address of replied packets. Anybody know some workaround? If anobody do not know some workaround can you confirm that it's not posible to do this with iptables?
read again the NAT part in the manual:
man iptables
nat table:
nat:
This table is consulted when a packet that creates a
new connection is encountered. It consists of three built-ins: PREROUTING (for altering packets as soon as they come in), OUTPUT (for altering locally-gener-
ated packets before routing), and POSTROUTING (for
altering packets as they are about to go out).
DNAT target:
DNAT
This target is only valid in the nat table, in the PREROUTING
and OUTPUT chains, and user-defined chains which are only called from those chains. It specifies that the destination address of the packet should be modified
(and all future packets in this connection will also be
mangled), and rules should cease being examined. It takes one type of option:
SNAT target:
SNAT
This target is only valid in the nat table, in the POSTROUTING
chain. It specifies that the source address of the packet should be modified (and all future packets in this connection will also be mangled), and rules should
cease being examined. It takes one type of option:
Quoted 9 lines
I've tried the next ip tables rules and only work when I do NEW connections from the web server.
-A POSTROUTING -o br0 -s
WE_SERVER_ADDR-p tcp -m tcp --sport 80 --dport 1024:65535 -j SNAT --to-sourceLOAD_BALANCER_ADDRThanks in advance.
Isaac González
You should do all of the NAT-ing ON THE LOAD BALANCER:
iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 80 --to-destination WEBSERVER1 (some load balancing options here) iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 80 --to-destination WEBSERVER2 (some load balancing options here)
iptables -t nat -A POSTROUTING -j SNAT -p tcp --dport 80 -d WEBSERVER1 --to-source BALANCER_IP_ON_WEBSERVER1_NET iptables -t nat -A POSTROUTING -j SNAT -p tcp --dport 80 -d WEBSERVER1 --to-source BALANCER_IP_ON_WEBSERVER2_NET
But some other rules may be in effect....
Swifty
Hi,
I know that doing the NAT in the firewall will do the trick, but the problem is that the "firewall and webserver" and the load balancer are in differents networks, then the webserver replies only goes through the firewall, and not though the load balancer. On the other hand the loadbalancer isn't a Linux box, then I can't not modify anything about packets, moreover I can't do any kind of routing.
VIP 10.0.0.10 |---LB---| LAN 80.67.12.X <<---ROUTERS---> FW 72.10.10.1 --> WS 72.10.10.10
HTTP REQUEST:
CUSTOMER IP 25.0.0.222 --> VIP 10.0.0.10
VIP 10.0.0.10 --> WS 72.10.10.10
WS 72.10.10.10 --> CUSTOMER 25.0.0.222
The CUSTOMER sees ACK that does'n not correspond with the original request (10.0.0.10) then the connection is not established.
I need a FW rule that change to source address of the webservers replies.
WS 72.10.10.10 --> VIP 10.0.0.10
Thanks in advance.
Quoted 60 lines · Gáspár Lajos
Hi,
2011-01-17 11:16 keltezéssel, GMail Isaac Gonzalez írta:
Quoted 4 lines
Hi,
I've doing some testing and seems that iptables only do SNAT on NEW connections, and I need to change the ip address of replied packets. Anybody know some workaround? If anobody do not know some workaround can you confirm that it's not posible to do this with iptables?
read again the NAT part in the manual:
man iptablesnat table:
nat:
This table is consulted when a packet that creates a
new connection is encountered. It consists of three built-ins: PREROUTING (for altering packets as soon as they come in), OUTPUT (for altering locally-gener-
ated packets before routing), and POSTROUTING (for
altering packets as they are about to go out).DNAT target:
DNAT
This target is only valid in the nat table, in the
PREROUTING and OUTPUT chains, and user-defined chains which are only called from those chains. It specifies that the destination address of the packet should be modified
(and all future packets in this connection will also be
mangled), and rules should cease being examined. It takes one type of option:SNAT target:
SNAT
This target is only valid in the nat table, in the POSTROUTING
chain. It specifies that the source address of the packet should be modified (and all future packets in this connection will also be mangled), and rules should
cease being examined. It takes one type of option:Quoted 9 lines
I've tried the next ip tables rules and only work when I do NEW connections from the web server.
-A POSTROUTING -o br0 -s
WE_SERVER_ADDR-p tcp -m tcp --sport 80 --dport 1024:65535 -j SNAT --to-sourceLOAD_BALANCER_ADDRThanks in advance.
Isaac González
You should do all of the NAT-ing ON THE LOAD BALANCER:
iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 80 --to-destination WEBSERVER1 (some load balancing options here) iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 80 --to-destination WEBSERVER2 (some load balancing options here)
iptables -t nat -A POSTROUTING -j SNAT -p tcp --dport 80 -d WEBSERVER1 --to-source
BALANCER_IP_ON_WEBSERVER1_NET
iptables -t nat -A POSTROUTING -j SNAT -p tcp --dport 80 -d WEBSERVER1 --to-sourceBALANCER_IP_ON_WEBSERVER2_NETBut some other rules may be in effect....
Swifty
Quoted 8 lines · GMail Isaac Gonzalez
Hi,
I know that doing the NAT in the firewall will do the trick, but the problem is that the "firewall and webserver" and the load balancer are in differents networks, then the webserver replies only goes through the firewall, and not though the load balancer. On the other hand the loadbalancer isn't a Linux box, then I can't not modify anything about packets, moreover I can't do any kind of routing.
Slow down, read again Gáspár response. Think particularly carefully about what the SNAT lines are doing there.
I'm fairly sure the LB will be capable of it somehow. If not, you have a nice heater to sit your feet on in winter.
As a giant hack you could also do SNAT on the device(s) receiving packets from the LB such that the reply packets get routed back through the LB.
Quoted 16 lines · Gáspár Lajos
Quoted 14 lines
Hi,
You should do all of the NAT-ing ON THE LOAD BALANCER:
iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 80 --to-destination WEBSERVER1 (some load balancing options here) iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 80 --to-destination WEBSERVER2 (some load balancing options here)
iptables -t nat -A POSTROUTING -j SNAT -p tcp --dport 80 -d WEBSERVER1 --to-source
BALANCER_IP_ON_WEBSERVER1_NET
iptables -t nat -A POSTROUTING -j SNAT -p tcp --dport 80 -d WEBSERVER1 --to-sourceBALANCER_IP_ON_WEBSERVER2_NET
Quoted 4 lines
Quoted 4 lines
But some other rules may be in effect....
Swifty
AYJ
Quoted 39 lines · Gáspár Lajos
Hi,
2011-01-17 11:16 keltezéssel, GMail Isaac Gonzalez írta:
Quoted 4 lines
Hi,
I've doing some testing and seems that iptables only do SNAT on NEW connections, and I need to change the ip address of replied packets. Anybody know some workaround? If anobody do not know some workaround can you confirm that it's not posible to do this with iptables?
read again the NAT part in the manual:
man iptablesnat table:
nat:
This table is consulted when a packet that creates a
new connection is encountered. It consists of three built-ins: PREROUTING (for altering packets as soon as they come in), OUTPUT (for altering locally-gener-
ated packets before routing), and POSTROUTING (for
altering packets as they are about to go out).DNAT target:
DNAT
This target is only valid in the nat table, in the
PREROUTING and OUTPUT chains, and user-defined chains which are only called from those chains. It specifies that the destination address of the packet should be modified
(and all future packets in this connection will also be
mangled), and rules should cease being examined. It takes one type of option:SNAT target:
SNAT
This target is only valid in the nat table, in the POSTROUTING
chain. It specifies that the source address of the packet should be modified (and all future packets in this connection will also be mangled), and rules should
cease being examined. It takes one type of option:Quoted 5 lines
I've tried the next ip tables rules and only work when I do NEW connections from the web server.
-A POSTROUTING -o br0 -s
WE_SERVER_ADDR-p tcp -m tcp --sport 80 --dport 1024:65535 -j SNAT --to-sourceLOAD_BALANCER_ADDR
Quoted 7 lines
Quoted 4 lines
Thanks in advance.
Isaac González
You should do all of the NAT-ing ON THE LOAD BALANCER:
I have to agree - if you are doing NAT you want to avoid any type of asymmetric routing - especially you NEED to make sure that the device that is doing the NAT (be it for load balancing or other reasons) receives the return packets. You cannot keep a TCP connection going if there is not some sort of state information being shared between the devices otherwise.
Signature · 9 lines
Best Regards, Giles Coochey NetSecSpec Ltd NL T-Systems Mobile: +31 681 265 086 NL Mobile: +31 626 508 131 Gib Mobile: +350 5401 6693 Email/MSN/Live Messenger: [email protected] Skype: gilescoochey
Ok,
Thanks for the replies, I was doing some tests to asure the connection issues. It's compulsory that the load balancer receive the return packets.
Quoted 50 lines · Giles Coochey
Quoted 39 lines · Gáspár Lajos
Hi,
2011-01-17 11:16 keltezéssel, GMail Isaac Gonzalez írta:
Quoted 4 lines
Hi,
I've doing some testing and seems that iptables only do SNAT on NEW connections, and I need to change the ip address of replied packets. Anybody know some workaround? If anobody do not know some workaround can you confirm that it's not posible to do this with iptables?
read again the NAT part in the manual:
man iptablesnat table:
nat:
This table is consulted when a packet that creates
a new connection is encountered. It consists of three built-ins: PREROUTING (for altering packets as soon as they come in), OUTPUT (for altering locally-gener-
ated packets before routing), and POSTROUTING (for
altering packets as they are about to go out).DNAT target:
DNAT
This target is only valid in the nat table, in the
PREROUTING and OUTPUT chains, and user-defined chains which are only called from those chains. It specifies that the destination address of the packet should be modified
(and all future packets in this connection will also be
mangled), and rules should cease being examined. It takes one type of option:SNAT target:
SNAT
This target is only valid in the nat table, in the POSTROUTING
chain. It specifies that the source address of the packet should be modified (and all future packets in this connection will also be mangled), and rules should
cease being examined. It takes one type of option:Quoted 5 lines
I've tried the next ip tables rules and only work when I do NEW connections from the web server.
-A POSTROUTING -o br0 -s
WE_SERVER_ADDR-p tcp -m tcp --sport 80 --dport 1024:65535 -j SNAT --to-sourceLOAD_BALANCER_ADDRQuoted 7 lines
Quoted 4 lines
Thanks in advance.
Isaac González
You should do all of the NAT-ing ON THE LOAD BALANCER:
I have to agree - if you are doing NAT you want to avoid any type of asymmetric routing - especially you NEED to make sure that the device that is doing the NAT (be it for load balancing or other reasons) receives the return packets. You cannot keep a TCP connection going if there is not some sort of state information being shared between the devices otherwise.
Quoted 7 lines · Giles Coochey
Quoted 3 lines
You should do all of the NAT-ing ON THE LOAD BALANCER:
I have to agree - if you are doing NAT you want to avoid any type of asymmetric routing - especially you NEED to make sure that the device that is doing the NAT (be it for load balancing or other reasons) receives the return packets.
Not strictly. You could utilize a second device whose CTs are synchronized with the LB to apply the reverse transform, using conntrackd. Sort of like
digraph { internet -> lb; lb -> web; web -> unnat; unnat -> internet; };
but it only looks feasible to me if your LB is already computationally crowded.
Quoted 10 lines · Jan Engelhardt
Quoted 5 lines · Giles Coochey
Quoted 2 lines
You should do all of the NAT-ing ON THE LOAD BALANCER:
I have to agree - if you are doing NAT you want to avoid any type of asymmetric routing - especially you NEED to make sure that the device that is doing the NAT (be it for load balancing or other reasons) receives the return packets.
Not strictly. You could utilize a second device whose CTs are synchronized with the LB to apply the reverse transform, using conntrackd. Sort of like
Quoted 5 lines
digraph { internet -> lb; lb -> web; web -> unnat; unnat -> internet; };
but it only looks feasible to me if your LB is already computationally crowded.
--
It also requires the loadbalancer to be using netfilter as well.
If it's a hardware load balancer with proprietary methods then you will need symmetric routing through it, unless it supports some form of TCP state sharing.
Signature · 9 lines
Best Regards, Giles Coochey NetSecSpec Ltd NL T-Systems Mobile: +31 681 265 086 NL Mobile: +31 626 508 131 Gib Mobile: +350 5401 6693 Email/MSN/Live Messenger: [email protected] Skype: gilescoochey
Quoted 11 lines · Giles Coochey
Quoted 10 lines · Jan Engelhardt
Quoted 5 lines · Giles Coochey
Quoted 2 lines
You should do all of the NAT-ing ON THE LOAD BALANCER:
I have to agree - if you are doing NAT you want to avoid any type of asymmetric routing - especially you NEED to make sure that the device that is doing the NAT (be it for load balancing or other reasons) receives the return packets.
Not strictly. You could utilize a second device whose CTs are synchronized with the LB to apply the reverse transform, using conntrackd. Sort of like
Quoted 9 lines
Quoted 5 lines
digraph { internet -> lb; lb -> web; web -> unnat; unnat -> internet; };
but it only looks feasible to me if your LB is already computationally crowded.
It also requires the loadbalancer to be using netfilter as well.
If it's a hardware load balancer with proprietary methods then you will need symmetric routing through it, unless it supports some form of TCP state sharing.
It works with propietary methods. The real solution if do symmetrical routing, all the replies must pass through the LB. I can't use conntrackd because I can't install anything in the LB.
Quoted 8 lines · GMail Isaac Gonzalez
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
NAT (be it for load balancing or other reasons) receives the return packets.
Not strictly. You could utilize a second device whose CTs are synchronized with the LB to apply the reverse transform, using conntrackd.
Quoted 5 lines
Quoted 1 line
It also requires the loadbalancer to be using netfilter as well.
It works with propietary methods. The real solution if do symmetrical routing, all the replies must pass through the LB. I can't use conntrackd because I can't install anything in the LB.
Too bad. That heater statement from earlier does have value ;-) ("Did you know?..." That Linux also has a load balancer included.)
Have you tried the RAWNAT modules from xtables-addons?
<snip>
iptables -A OUTPUT -s $ORIG_IP -j RAWSNAT --to-source $NEW_IP </snip>
I've used RAWSNAT in the past, with fairly good success.
-Mike
Quoted 32 lines · On Mon, 17 Jan 2011 11:16:48 +0100 GMail Isaac Gonzalez…
Hi,
I need to modify the reply packets of one web server to allow the connections between a webserver and client using a load balancer.
The client connections goes to a load balancer, the load balancer forwards the connection to a one web server changing the destination ip, the web server anwser the client with it's own ip address without passing again for the load balancer. In order to stablish the connection, the client needs to receive the web server answer with the correct ip address (in this case, the load balancer VIP address), in other case it receives ACK that it doesn't know about it and the connections is not ESTABLISHED.
I've doing some testing and seems that iptables only do SNAT on NEW connections, and I need to change the ip address of replied packets. Anybody know some workaround? If anobody do not know some workaround can you confirm that it's not posible to do this with iptables?
I've tried the next ip tables rules and only work when I do NEW connections from the web server.
-A POSTROUTING -o br0 -s
WE_SERVER_ADDR-p tcp -m tcp --sport 80 --dport 1024:65535 -j SNAT --to-sourceLOAD_BALANCER_ADDRThanks in advance.
Isaac González
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 1 line
Michael Vallaly <[email protected]>