iptables NAT routing issues
Hello,
We're having some problems with iptables, have been playing ~3 hours with this and I need some advice.
We want to 'forward' port 80 from one ip to another ip (other server) - mainly to protect the webserver. One could say that you could use basic portforwarding for this, but as far as i know this also breaks the source ip for apache logs, etc. So we decided that we needed NAT'ing.
On the 'outbound' server we're doing DNAT to the webserver, which seems to work fine (can see the packets in tcpdump on web), and on the webserver we do SNAT back to the 'original' (outboundserver) ip the user connected to.
The last part does not work tho, the webserver still uses its own outgoing ip, and not the IP we specified. If I telnet to some host however, using the webserver source ip - it DOES rewrite the source address (and fails to connect, of course).
Tried various iptables commands on the servers:
OUTBOUND SERVER:
iptables -t nat -A PREROUTING -p tcp --dport 80 -d <ext_web_ip> -j DNAT --to-destination <webserver_ip>:80
Note: We didn't touch this as it seems to works fine.
WEBSERVER:
Tried various, from the detailed:
iptables -t nat -A POSTROUTING -s <webserver_ip>-p tcp --sport 80 -j
SNAT --to-source <ext_web_ip>:80 -o eth0
To the 'should-be-catching-it-all':
iptables -t nat -A POSTROUTING -s <webserver_ip> -j SNAT --to-source
<ext_web_ip>
None of those seem to do the thing we want to do tho.
Some advice would be very welcome.
Cheers,
Bas Verhoeven
Quoted 9 lines · Bas Verhoeven
OUTBOUND SERVER:
iptables -t nat -A PREROUTING -p tcp --dport 80 -d <
ext_web_ip> -j DNAT --to-destination <webserver_ip>:80Note: We didn't touch this as it seems to works fine.
WEBSERVER:
You do not need any rules here. The outbound server will do the translation in both directions.
Quoted 8 lines
Tried various, from the detailed:
iptables -t nat -A POSTROUTING -s <webserver_ip>-p tcp --sport 80 -j SNAT
--to-source <ext_web_ip>:80 -o eth0To the 'should-be-catching-it-all':
iptables -t nat -A POSTROUTING -s <webserver_ip> -j SNAT --to-source
<ext_web_ip>
Quoted 7 lines
None of those seem to do the thing we want to do tho.
Some advice would be very welcome.
Cheers,
Bas Verhoeven
Jan
Hello,
Quoted 7 lines · Jan Engelhardt
Quoted 5 lines · Bas Verhoeven
OUTBOUND SERVER:
iptables -t nat -A PREROUTING -p tcp --dport 80 -d <
ext_web_ip> -j DNAT --to-destination <webserver_ip>:80
Quoted 6 lines
Quoted 3 lines
Note: We didn't touch this as it seems to works fine.
WEBSERVER:
You do not need any rules here. The outbound server will do the translation in both directions.
Assuming that the outbound server is the default gateway for the web server and receives the return traffic, of course.
Quoted 10 lines · Pascal Hambourg
Jan Engelhardt a écrit :
Quoted 9 lines · Bas Verhoeven
Quoted 8 lines
OUTBOUND SERVER:
iptables -t nat -A PREROUTING -p tcp --dport 80 -d <
ext_web_ip> -j DNAT --to-destination <webserver_ip>:80Note: We didn't touch this as it seems to works fine.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
WEBSERVER:
You do not need any rules here. The outbound server will do the translation
in both directions.Assuming that the outbound server is the default gateway for the web server and receives the return traffic, of course.
You'd be kinda screwed if that was not the case. Because the 'nat' table is only consulted for NEW connections afaics, the reply packets do _not_ get SNATed, since the _first_ packet was the TCP SYN, which makes the connection ESTABLISHED when input processing is done.
Jan
Quoted 11 lines · Jan Engelhardt
Quoted 4 lines
Assuming that the outbound server is the default gateway for the web server and receives the return traffic, of course.
You'd be kinda screwed if that was not the case. Because the 'nat' table is only consulted for NEW connections afaics, the reply packets do _not_ get SNATed, since the _first_ packet was the TCP SYN, which makes the connection ESTABLISHED when input processing is done.
The webserver has its own gateway, and that's not the outer box. You seem to be describing the exact problem i'm seeing in tcpdump; new connections get snat'ed, 'forwarded' connections do not.
Is there really no solution for this?
Bas
Am Wednesday, den 9 May hub Bas Verhoeven folgendes in die Tasten:
Quoted 13 lines
Quoted 10 lines · Jan Engelhardt
Quoted 5 lines
Assuming that the outbound server is the default gateway for the web server and
receives the return traffic, of course.You'd be kinda screwed if that was not the case. Because the 'nat' table is only consulted for NEW connections afaics, the reply packets do _not_ get SNATed, since the _first_ packet was the TCP SYN, which makes the connection ESTABLISHED when input processing is done.
Quoted 5 lines
The webserver has its own gateway, and that's not the outer box. You seem to be describing the exact problem i'm seeing in tcpdump; new connections get snat'ed, 'forwarded' connections do not.
Is there really no solution for this?
The solution is to use OUTBOUND SERVER as default gateway or do the DNATing on the gateway.
Or stop DNATing and use "normal" rules to only allow traffic to WEBSERVER 80/tcp and reject the remaining traffic.
Ciao
Max
Signature · 1 line
Follow the white penguin.
Quoted 6 lines · Bas Verhoeven
Quoted 4 lines
Quoted 2 lines
Assuming that the outbound server is the default gateway for the web server and receives the return traffic, of course.
You'd be kinda screwed if that was not the case.
The webserver has its own gateway, and that's not the outer box.
Don't look further. Here is why it does not work.
Quoted 1 line
Is there really no solution for this?
Use the outer box as a gateway, if it is in the same network. You do not have to use it as the default gateway for all traffic but at least for the HTTP return traffic. This could be done with iptables and advanced routing on the web server, for instance using the source port 80 to MARK packets or using CONNMARK/connmark.
You could try to use the NOTRACK target on the web server in order to prevent the incoming SYN packets from creating a new connection, so the replies could be SNATed. But I would not bet a euro-cent on such a dirty method. If someone asks who suggested it, don't say it's me. ;-)
Quoted 1 line · Pascal Hambourg
Use the outer box as a gateway, if it is in the same network. You do not have to use it as the default gateway for all traffic but at least for the HTTP return traffic. This could be done with iptables and advanced routing on the web server, for instance using the source port 80 to MARK packets or using CONNMARK/connmark.
I wasn't aware of that option, it seems to work fine ;-)
Thanks.
Quoted 3 lines
You could try to use the NOTRACK target on the web server in order to prevent the incoming SYN packets from creating a new connection, so the replies could be SNATed. But I would not bet a euro-cent on such a dirty method. If someone asks who suggested it, don't say it's me. ;-)
Quoted 4 lines · Bas Verhoeven
Quoted 2 lines
Use the outer box as a gateway, if it is in the same network. You do not have to use it as the default gateway for all traffic but at least for the HTTP return traffic. This could be done with iptables and advanced routing on the web server, for instance using the source port 80 to MARK packets or using CONNMARK/connmark.
I wasn't aware of that option, it seems to work fine ;-)
Which option did you choose ?
Quoted 1 line
Thanks.
You're welcome.
Quoted 1 line · Pascal Hambourg
Which option did you choose ?
We still have the outbound server DNAT-ing connections to the webserver's ip, that worked fine. On the webserver we now mark all outgoing web packets:
# iptables -t mangle -A OUTPUT -s <webserver_inner_ip> -p tcp --sport 80 -j MARK --set-mark 2
And we use iproute2 to forward them back to the outbound server:
# ip rule add fwmark 2 pref 10 table web.out # ip route add default via <outbound_box_ip> dev eth0 table web.out
Couldn't test with CONNMARK, as the box doesn't ship with that, but MARK works great for now.
I did test your last option too, but that just didn't work and sounded very hacky-ish, not something we could rely on, even if it worked.
Quoted 2 lines · Bas Verhoeven
Quoted 2 lines
Which option did you choose ?
[...]
Quoted 1 line
On the webserver we now mark all outgoing web packets:
[...]
Quoted 1 line
And we use iproute2 to forward them back to the outbound server:
[...]
Ok. You seem to know how to use advanced routing, I am a bit surprised you said you "weren't aware of that option".
Quoted 1 line
Couldn't test with CONNMARK, as the box doesn't ship with that, but MARK works great for now.
The 'CONNMARK' target and the 'connmark' match were included in the kernel 2.6.10. Earlier kernel versions had to be patched with the patch-o-matic-ng.
Quoted 1 line
I did test your last option too, but that just didn't work and sounded very hacky-ish, not something we could rely on, even if it worked.
Huh, what last option ? I don't know what you're talking about. ;-)
Am Tuesday, den 8 May hub Bas Verhoeven folgendes in die Tasten:
Hi!
Quoted 7 lines
We're having some problems with iptables, have been playing ~3 hours with this and I need some advice.
We want to 'forward' port 80 from one ip to another ip (other server) - mainly to protect the webserver. One could say that you could use basic portforwarding for this, but as far as i know this also breaks the source ip for apache logs, etc. So we decided that we needed NAT'ing.
[...]
Quoted 4 lines
OUTBOUND SERVER:
iptables -t nat -A PREROUTING -p tcp --dport 80 -d <
ext_web_ip> -j DNAT --to-destination <webserver_ip>:80
That´s fine.
This will also make netfilter care of answer packages.
Quoted 3 lines
Note: We didn't touch this as it seems to works fine.
WEBSERVER:
[...]
There are no NAT rules needed here.
All you have to accomplish is that the answer packages from WEBSERVER to $client are routed via OUTBOUND SERVER.
I guess that´s just true by the network architecture.
e.g.
client network / inet <---> OUTBOUND SERVER <-> WEBSERVER
HTH
Ciao
Max
Signature · 1 line
Follow the white penguin.