"Proper" way to transparent proxy?
Short story: I need to write a transparent, application level proxy. I've searched through the mailing lists for information on how to do it, and find conflicting information about things like "stateless redirection" patches by Bazsi, and comments that they would no longer be maintained because something better was coming. But I can't tell if that something better happened, or what it is.
If there's something I should be looking at, I'd be greatful for pointers.
Long story: we have an OpenBSD based firewall that uses this particular application level proxy, using IPFilter's transparent proxy support. We are investigating rehosting the whole thing on Linux (for the record, MY first choice anyways :) and this is the hardest problem so far:
I need to find the address and port the outgoing connecion was originally heading to. I'm assuming this is what's meant by the stateless redirection patches and such, but those (admittedly) didn't even come with documentation.
I'm starting from scratch here: I've never programmed to Netfilter before. What I need is fairly simple, I think, but I don't know how to get started.
What we had on IPFilter was a "redirect" rule that redirected all connections to external hosts on this specific port to a specific port on the firewall machine, where a proxy program was waiting for it. We then used IPFilter system calls to retreive where the connection was heading.
I expect that I'll be able to reuse most of my code: just the lookup part needs to change.
Thanks in advance for any help/pointers. I'll attempt to collect what I've learned and post it, so that it ends up in the archives and more people like me don't have to ask. :)
Kevin
Quoted 5 lines · Kevin White
What we had on IPFilter was a "redirect" rule that redirected all connections to external hosts on this specific port to a specific port on the firewall machine, where a proxy program was waiting for it. We then used IPFilter system calls to retreive where the connection was heading.
This is exactly how it works on linux, you redirect (or reroute+redirect if on different machines), accept the connection and call a sysctl to find out real destination. As for examples, there are several open source transparent proxies:
- squid
- tircproxy
- jftpgw
Quoted 1 line
Kevin
Bye,
Peter Surda (Shurdeek) <[email protected]>, ICQ 10236103, +436505122023
--
"Where do you want to go to die?"
Quoted 2 lines
I need to find the address and port the outgoing connecion was originally heading to.
Use standard iptables, with a commandline like this:
iptables -t nat -p tcp --dport 80 -j REDIRECT --to-port 1234
Quoted 2 lines
I expect that I'll be able to reuse most of my code: just the lookup part needs to change.
Once upon at time, in Linux 2.0.x, with libc5, you had getsockname() return TheRightThing - in this particular situation. This is no longer the case. Nowadays, with iptables and a rule like the one above, you call getsockopt(), querying for SO_ORIGINAL_DST.
google tells me, giving it that hint, that no better documentation than some ooooold mail threads exists on SO_ORIGINAL_DST. My bad. I hate it when mailing list traffic by me is first hit :)
For the benefit of the eternal (hope hope hope) google, I am using the exact code below, compiled like this:
gcc -c preload_getsockname.c
ld -o preload_getsockname.so -shared preload_getsockname.o -ldl
as an LD_PRELOAD file for a very ancient squid binary. In production on some handful on boxes. You probably only need the 'getsockopt' line.
It works(tm).
best regards
Patrick
#include <sys/socket.h>
#include <dlfcn.h>
#ifndef SO_ORIGINAL_DST
#include <linux/netfilter_ipv4.h>
#ifndef SO_ORIGINAL_DST
#error no SO_ORIGINAL_DST defined
#endif
#endif
int getsockname(int fd, struct sockaddr *sa, int *salen)
{
static int (*libc_getsockname)(int, struct sockaddr *, int *);
int ret;
if (!libc_getsockname) {
libc_getsockname = (int (*)(int, struct sockaddr *, int
*))
dlsym(RTLD_NEXT, "getsockname");
}
ret = getsockopt(fd, SOL_IP, SO_ORIGINAL_DST, sa, salen);
if (ret < 0 && libc_getsockname)
ret = libc_getsockname(fd, sa, salen);
return ret;
}how to flush ip conntrack entries ?
i run into the problem described here and i wondered if someone knows of a solution:
+----------+
| SNAT |
private|--------> |(eth1: public ip)
ip +----------+1) setup masquerading
(iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADING)
and ping continuously from private to public. all ok.
2) flush the NAT
iptables -t nat -F
(ping will stop working obviously but i don't kill the ping process, just keep sending echo, no reply)
3) put back nat:
(iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADING)
!!! ping does not come back !!!
4) I have to stop the pings and
5) restart them to make it work.
looking at the ip_conntrack proc entry it was noticed that:
after flushing (step 2) an UNREPLIED entry for icmp is there (no reply hence unreplied) but its ttl does not decrement. (ping echos are still hitting the nat box from private side)
stoping the ping (step 4) allows the ttl timer of the conntrack entry to start decrementing (30 sec)
restaring the pings (i don't have to wait till ttl goes to zero ?!?) (step 5) but now with nat back on (step 3) I don;t get the icmp entry on conntrack but all is ok (pings goes thru).
question is: is there a way to achieve this (looks like start decrementing that ttl or reseting it to zero in conntrack) in the nat box without having to stop the pings on the host side ?
thanks,
marian
Quoted 8 lines
For the benefit of the eternal (hope hope hope) google, I am using the exact code below, compiled like this:
gcc -c
preload_getsockname.c
ld -o preload_getsockname.so -shared preload_getsockname.o -ldlas an
LD_PRELOADfile for a very ancient squid binary. In production on some handful on boxes. You probably only need the 'getsockopt' line.
Great...thanks tons. I'll put it into action...
Thanks again!
Kevin
Re: how to flush ip conntrack entries ?
In reply to marian stagarescuQuoted 5 lines · marian stagarescu
looking at the
ip_conntrackproc entry it was noticed that:after flushing (step 2) an UNREPLIED entry for icmp is there (no reply hence unreplied) but its ttl does not decrement. (ping echos are still hitting the nat box from private side)
Normally icmp conntrack entries are deleted as soon as there is a reply. Once you remove the MASQ rule though, there is no reply, so the conntrack stays around. Any subsequent packets from the same ping process will match this conntrack, and thus be NATed exactly the same way. Adding the MASQ rule back in does not affect the existing conntrack.
Quoted 1 line
stoping the ping (step 4) allows the ttl timer of the conntrack entry to start decrementing (30 sec)
The timer doesn't seem to decrement while the ping is still going because the ping packets are matching the conntrack and refreshing the timer back to 30 seconds. Stop the ping and the timer is no longer refreshed.
Quoted 3 lines
restaring the pings (i don't have to wait till ttl goes to zero ?!?) (step 5) but now with nat back on (step 3) I don;t get the icmp entry on conntrack but all is ok (pings goes thru).
When you start a new ping process, the ping packets have a new id, and so they don't match the old conntrack. A new conntrack is created, and the new MASQ rule is used to masquerade them correctly. You get a ping reply now, and so the conntrack is immediately deleted, which is why you don't see it.
Quoted 3 lines
question is: is there a way to achieve this (looks like start decrementing that ttl or reseting it to zero in conntrack) in the nat box without having to stop the pings on the host side ?
You can flush conntrack entries for masqueraded connections by doing either a down/up or ip addr add/del on the associated interface. This probably won't help you in this case though because the problem conntrack is not masqueraded. I don't know of any other ways of flushing conntracks.
Regards
Signature · 3 lines
Philip Craig Software Engineer http://www.SnapGear.com [email protected] Ph: +61 7 3435 2821 Fx: +61 7 3891 3630 SnapGear - Custom Embedded Solutions and Security Appliances
Re: how to flush ip conntrack entries ?
hi philip,
thanks for you answer.
yes ifconfig up/down works but i wanted to make it work without doing this.
i noticed that the ipsysctl tutorial 1.0.0 (posted to the list by askar) documentation points to a /proc/sys/net/ipv4/netfilter entry with an ip_ct_generic_timeout
and ip_ct_icmp_timeout
i don't have those into my kernel (2.4.17 patched with newnat13 and userland iptables 1.2.6a), maybe someone on the list can tell me which patch installs these entries.
having those i can try to set to zero the above ip_ct_icmp_timeout
marian
Quoted 47 lines · Philip Craig
Quoted 5 lines · marian stagarescu
looking at the
ip_conntrackproc entry it was noticed that:after flushing (step 2) an UNREPLIED entry for icmp is there (no reply hence unreplied) but its ttl does not decrement. (ping echos are still hitting the nat box from private side)
Normally icmp conntrack entries are deleted as soon as there is a reply. Once you remove the MASQ rule though, there is no reply, so the conntrack stays around. Any subsequent packets from the same ping process will match this conntrack, and thus be NATed exactly the same way. Adding the MASQ rule back in does not affect the existing conntrack.
Quoted 2 lines
stoping the ping (step 4) allows the ttl timer of the conntrack entry to start decrementing (30 sec)
The timer doesn't seem to decrement while the ping is still going because the ping packets are matching the conntrack and refreshing the timer back to 30 seconds. Stop the ping and the timer is no longer refreshed.
Quoted 3 lines
restaring the pings (i don't have to wait till ttl goes to zero ?!?) (step 5) but now with nat back on (step 3) I don;t get the icmp entry on conntrack but all is ok (pings goes thru).
When you start a new ping process, the ping packets have a new id, and so they don't match the old conntrack. A new conntrack is created, and the new MASQ rule is used to masquerade them correctly. You get a ping reply now, and so the conntrack is immediately deleted, which is why you don't see it.
Quoted 3 lines
question is: is there a way to achieve this (looks like start decrementing that ttl or reseting it to zero in conntrack) in the nat box without having to stop the pings on the host side ?
You can flush conntrack entries for masqueraded connections by doing either a down/up or ip addr add/del on the associated interface. This probably won't help you in this case though because the problem conntrack is not masqueraded. I don't know of any other ways of flushing conntracks.
Regards
Signature · 3 lines
Philip Craig Software Engineer http://www.SnapGear.com [email protected] Ph: +61 7 3435 2821 Fx: +61 7 3891 3630 SnapGear - Custom Embedded Solutions and Security Appliances
Re: how to flush ip conntrack entries ?
i see these entries under tcp window tracking patch i'll give it a try.
Quoted 69 lines · marian stagarescu
hi philip,
thanks for you answer.yes ifconfig up/down works but i wanted to make it work without doing this.
i noticed that the ipsysctl tutorial 1.0.0 (posted to the list by askar) documentation points to a
/proc/sys/net/ipv4/netfilterentry with anip_ct_generic_timeout
andip_ct_icmp_timeout
i don't have those into my kernel (2.4.17 patched with newnat13 and userland iptables 1.2.6a), maybe someone on the list can tell me which patch installs these entries.
having those i can try to set to zero the aboveip_ct_icmp_timeoutmarian
Quoted 47 lines · Philip Craig
Quoted 5 lines · marian stagarescu
looking at the
ip_conntrackproc entry it was noticed that:after flushing (step 2) an UNREPLIED entry for icmp is there (no reply hence unreplied) but its ttl does not decrement. (ping echos are still hitting the nat box from private side)
Normally icmp conntrack entries are deleted as soon as there is a reply. Once you remove the MASQ rule though, there is no reply, so the conntrack stays around. Any subsequent packets from the same ping process will match this conntrack, and thus be NATed exactly the same way. Adding the MASQ rule back in does not affect the existing conntrack.
Quoted 2 lines
stoping the ping (step 4) allows the ttl timer of the conntrack entry to start decrementing (30 sec)
The timer doesn't seem to decrement while the ping is still going because the ping packets are matching the conntrack and refreshing the timer back to 30 seconds. Stop the ping and the timer is no longer refreshed.
Quoted 3 lines
restaring the pings (i don't have to wait till ttl goes to zero ?!?) (step 5) but now with nat back on (step 3) I don;t get the icmp entry on conntrack but all is ok (pings goes thru).
When you start a new ping process, the ping packets have a new id, and so they don't match the old conntrack. A new conntrack is created, and the new MASQ rule is used to masquerade them correctly. You get a ping reply now, and so the conntrack is immediately deleted, which is why you don't see it.
Quoted 3 lines
question is: is there a way to achieve this (looks like start decrementing that ttl or reseting it to zero in conntrack) in the nat box without having to stop the pings on the host side ?
You can flush conntrack entries for masqueraded connections by doing either a down/up or ip addr add/del on the associated interface. This probably won't help you in this case though because the problem conntrack is not masqueraded. I don't know of any other ways of flushing conntracks.
Regards
Signature · 3 lines
Philip Craig Software Engineer http://www.SnapGear.com [email protected] Ph: +61 7 3435 2821 Fx: +61 7 3891 3630 SnapGear - Custom Embedded Solutions and Security Appliances