Skip to contentSkip to conversation

Transparent proxying patches version 4

PATCH In review KOVACS Krisztian 11 participants 47 messages Jan 3, 2007 – Feb 7, 2007

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

cover Transparent proxying patches version 4

The following set of patches implement transparent proxying support loosely modeled on the Linux 2.2 transparent proxying functionality.

In the last few years we've been maintaining a set of patches implementing Netfilter NAT to provide similar functionality. However, as time passed, more and more bugs surfaced, some of which were not possible to fix using that approach. Also, those patches required modification of user-space application code and the "API" provided was neither clean nor easy to use.

So instead of using NAT to dynamically redirect traffic to local addresses, we now rely on "native" non-locally-bound sockets and do early socket lookups for inbound IPv4 packets. These lookups are done in a separate Netfilter/iptables module, so there are only negligible performance implications of building transparent proxying support as a module and then not loading it.

Small modifications were also necessary in IP/TCP/UDP core code to support the Netfilter modules. All those have been functionally split out into stand-alone patches among which there are no direct dependencies. Among these changes are ones which I think might be potentially risky, especially the core IPv4 routing code changes.

Also please note that at the moment only IPv4 support is implemented, but opposed to the NAT-based approach taken by older TProxy versions IPv6 support is possible this way.

Comments welcome...

Signature · 2 lines
 Regards,
  Krisztian Kovacs
#1 lore raw

PATCH 1/10 Implement local diversion of IPv4 skbs

The input path for non-local bound sockets requires diverting certain packets locally, even if their destination IP address is not considered local. We achieve this by assigning a specially crafted dst entry to these skbs, and optionally also attaching a socket to the skb so that the upper layer code does not need to redo the socket lookup.

We also have to be able to differentiate between these fake entries and "real" entries in the cache: it is perfectly legal that the diversion is done only for certain TCP or UDP packets and not for all packets of the flow. Since these special dst entries are used only by the iptables tproxy code, and that code uses exclusively these entries, simply flagging these entries as DST_DIVERTED is OK. All other cache lookup paths skip diverted entries, while our new ip_divert_local() function uses exclusively diverted dst entries.

---

 include/net/dst.h   |    1 
 include/net/route.h |    2 +
 net/ipv4/route.c    |  106 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 108 insertions(+), 1 deletions(-)
include/net/dst.h +1 -0
index 62b7e75..72b712c 100644
@@ -50,6 +50,7 @@ #define DST_NOXFRM		2
 #define DST_NOPOLICY		4
 #define DST_NOHASH		8
 #define DST_BALANCED            0x10
+#define DST_DIVERTED		0x20
 	unsigned long		lastuse;
 	unsigned long		expires;
 
include/net/route.h +2 -0
index 486e37a..ee52393 100644
@@ -126,6 +126,8 @@ extern int		ip_rt_ioctl(unsigned int cmd
 extern void		ip_rt_get_source(u8 *src, struct rtable *rt);
 extern int		ip_rt_dump(struct sk_buff *skb,  struct netlink_callback *cb);
 
+extern int		ip_divert_local(struct sk_buff *skb, const struct in_device *in, struct sock *sk);
+
 struct in_ifaddr;
 extern void fib_add_ifaddr(struct in_ifaddr *);
 
net/ipv4/route.c +105 -1

135 lines not on this page · raw message

#2 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

Quoted 100 lines · KOVACS Krisztian

The input path for non-local bound sockets requires diverting certain packets locally, even if their destination IP address is not considered local. We achieve this by assigning a specially crafted dst entry to these skbs, and optionally also attaching a socket to the skb so that the upper layer code does not need to redo the socket lookup.

We also have to be able to differentiate between these fake entries and "real" entries in the cache: it is perfectly legal that the diversion is done only for certain TCP or UDP packets and not for all packets of the flow. Since these special dst entries are used only by the iptables tproxy code, and that code uses exclusively these entries, simply flagging these entries as DST_DIVERTED is OK. All other cache lookup paths skip diverted entries, while our new ip_divert_local() function uses exclusively diverted dst entries.

---

 include/net/dst.h   |    1 
 include/net/route.h |    2 +
 net/ipv4/route.c    |  106 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 108 insertions(+), 1 deletions(-)
include/net/dst.h +1 -0
index 62b7e75..72b712c 100644
@@ -50,6 +50,7 @@ #define DST_NOXFRM		2
 #define DST_NOPOLICY		4
 #define DST_NOHASH		8
 #define DST_BALANCED            0x10
+#define DST_DIVERTED		0x20
 	unsigned long		lastuse;
 	unsigned long		expires;
 
include/net/route.h +2 -0
index 486e37a..ee52393 100644
@@ -126,6 +126,8 @@ extern int		ip_rt_ioctl(unsigned int cmd
 extern void		ip_rt_get_source(u8 *src, struct rtable *rt);
 extern int		ip_rt_dump(struct sk_buff *skb,  struct netlink_callback *cb);
 
+extern int		ip_divert_local(struct sk_buff *skb, const struct in_device *in, struct sock *sk);
+
 struct in_ifaddr;
 extern void fib_add_ifaddr(struct in_ifaddr *);
 
net/ipv4/route.c +26 -1
index 2daa0dc..537b976 100644
@@ -942,9 +942,11 @@ restart:
 	while ((rth = *rthp) != NULL) {
 #ifdef CONFIG_IP_ROUTE_MULTIPATH_CACHED
 		if (!(rth->u.dst.flags & DST_BALANCED) &&
+		    ((rt->u.dst.flags & DST_DIVERTED) == (rth->u.dst.flags & DST_DIVERTED)) &&
 		    compare_keys(&rth->fl, &rt->fl)) {
 #else
-		if (compare_keys(&rth->fl, &rt->fl)) {
+		if (((rt->u.dst.flags & DST_DIVERTED) == (rth->u.dst.flags & DST_DIVERTED)) &&
+		    compare_keys(&rth->fl, &rt->fl)) {
 #endif
 			/* Put it first */
 			*rthp = rth->u.rt_next;
@@ -1166,6 +1168,7 @@ void ip_rt_redirect(__be32 old_gw, __be3
 				if (rth->fl.fl4_dst != daddr ||
 				    rth->fl.fl4_src != skeys[i] ||
 				    rth->fl.oif != ikeys[k] ||
+				    (rth->u.dst.flags & DST_DIVERTED) ||
 				    rth->fl.iif != 0) {
 					rthp = &rth->u.rt_next;
 					continue;
@@ -1526,6 +1529,105 @@ static int ip_rt_bug(struct sk_buff *skb
 	return 0;
 }
 
+static void ip_divert_free_sock(struct sk_buff *skb)
+{
+	struct sock *sk = skb->sk;
+
+	skb->sk = NULL;
+	skb->destructor = NULL;
+	sock_put(sk);
+}
+
+int ip_divert_local(struct sk_buff *skb, const struct in_device *in, struct sock *sk)
+{
+	struct iphdr *iph = skb->nh.iph;
+	struct rtable *rth, *rtres;
+	unsigned hash;
+	const int iif = in->dev->ifindex;
+	u_int8_t tos;
+	int err;
+
+	/* look up hash first */
+	tos = iph->tos & IPTOS_RT_MASK;
+	hash = rt_hash_code(iph->daddr, iph->saddr ^ (iif << 5));
+

net/ipv4/route.c static int ip_rt_bug(struct sk_buff *skb · line 1563

Quoted 10 lines
net/ipv4/route.c +10 -0
+	rcu_read_lock();
+	for (rth = rcu_dereference(rt_hash_table[hash].chain); rth;
+	     rth = rcu_dereference(rth->u.rt_next)) {
+		if (rth->fl.fl4_dst == iph->daddr &&
+		    rth->fl.fl4_src == iph->saddr &&
+		    rth->fl.iif == iif &&
+		    rth->fl.oif == 0 &&
+		    rth->fl.mark == skb->mark &&
+		    (rth->u.dst.flags & DST_DIVERTED) &&
+		    rth->fl.fl4_tos == tos) {

Mark and tos look unnecessary here since they don't affect the further processing of the packet.

net/ipv4/route.c static int ip_rt_bug(struct sk_buff *skb · line 1575

Quoted 12 lines
net/ipv4/route.c +12 -0
+			rth->u.dst.lastuse = jiffies;
+			dst_hold(&rth->u.dst);
+			rth->u.dst.__use++;
+			RT_CACHE_STAT_INC(in_hit);
+			rcu_read_unlock();
+
+			dst_release(skb->dst);
+			skb->dst = (struct dst_entry*)rth;
+
+			if (sk) {
+				sock_hold(sk);
+				skb->sk = sk;

This looks racy, the socket could be closed between the lookup and the actual use. Why do you need the socket lookup at all, can't you just divert all packets selected by iptables?

I'm wondering if it would be possible to use normal input routing combined with netfilter marks to do the diversion ..

Quoted 72 lines
net/ipv4/route.c +57 -0
+				skb->destructor = ip_divert_free_sock;
+			}
+
+			return 0;
+		}
+		RT_CACHE_STAT_INC(in_hlist_search);
+	}
+	rcu_read_unlock();
+
+	/* not found in cache, try to allocate a new dst entry */
+	rth = dst_alloc(&ipv4_dst_ops);
+	if (!rth)
+		return -ENOMEM;
+
+	rth->u.dst.output= ip_rt_bug;
+
+	atomic_set(&rth->u.dst.__refcnt, 1);
+	rth->u.dst.flags = DST_HOST | DST_DIVERTED;
+
+	if (in->cnf.no_policy)
+		rth->u.dst.flags |= DST_NOPOLICY;
+
+	rth->fl.fl4_dst = iph->daddr;
+	rth->rt_dst	= iph->daddr;
+	rth->fl.fl4_tos = iph->tos;
+	rth->fl.mark	= skb->mark;
+	rth->fl.fl4_src = iph->saddr;
+	rth->rt_src	= iph->saddr;
+	rth->rt_iif	=
+	rth->fl.iif	= skb->dev->ifindex;
+	rth->u.dst.dev	= &loopback_dev;
+	dev_hold(rth->u.dst.dev);
+	rth->idev	= in_dev_get(rth->u.dst.dev);
+	rth->rt_gateway = iph->daddr;
+	rth->rt_spec_dst= iph->daddr;
+	rth->u.dst.input= ip_local_deliver;
+	rth->rt_flags	= RTCF_LOCAL;
+	rth->rt_type	= RTN_LOCAL;
+
+	err = rt_intern_hash(hash, rth, &rtres);
+	if (err)
+		return err;
+
+	dst_release(skb->dst);
+	skb->dst = (struct dst_entry *) rth;
+
+	if (sk) {
+		sock_hold(sk);
+		skb->sk = sk;
+		skb->destructor = ip_divert_free_sock;
+	}
+
+	return 0;
+}
+
 /*
    We do not cache source address of outgoing interface,
    because it is used only by IP RR, TS and SRR options,
@@ -2104,6 +2206,7 @@ int ip_route_input(struct sk_buff *skb,
 		    rth->fl.fl4_src == saddr &&
 		    rth->fl.iif == iif &&
 		    rth->fl.oif == 0 &&
+		    !(rth->u.dst.flags & DST_DIVERTED) &&
 		    rth->fl.mark == skb->mark &&
 		    rth->fl.fl4_tos == tos) {
 			rth->u.dst.lastuse = jiffies;
@@ -3199,3 +3302,4 @@ #endif
 EXPORT_SYMBOL(__ip_select_ident);
 EXPORT_SYMBOL(ip_route_input);
 EXPORT_SYMBOL(ip_route_output_key);
+EXPORT_SYMBOL_GPL(ip_divert_local);
#3 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

Quoted 19 lines · Patrick McHardy

net/ipv4/route.c static int ip_rt_bug(struct sk_buff *skb · line 1575

Quoted 12 lines · KOVACS Krisztian
net/ipv4/route.c +12 -0
+			rth->u.dst.lastuse = jiffies;
+			dst_hold(&rth->u.dst);
+			rth->u.dst.__use++;
+			RT_CACHE_STAT_INC(in_hit);
+			rcu_read_unlock();
+
+			dst_release(skb->dst);
+			skb->dst = (struct dst_entry*)rth;
+
+			if (sk) {
+				sock_hold(sk);
+				skb->sk = sk;

This looks racy, the socket could be closed between the lookup and the actual use. Why do you need the socket lookup at all, can't you just divert all packets selected by iptables?

Quoted 3 lines

I'm wondering if it would be possible to use normal input routing combined with netfilter marks to do the diversion ..

The problem is that userspace proxies open ports dynamically (think of FTP data channels), you cannot add iptables rule for every such redirection. So one rule for every dynamic redirection is a no-go.

If we'd add a single rule, which would do some kind of lookup and then mark packets, would again introduce a state inside tproxy that'd need to be synchronized with the socket table. We explicitly wanted to avoid such tables.

And additionally, using the mark this way would prevent the admin to use it they way he/she likes.

Signature · 1 line
Bazsi
#4 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

Quoted 6 lines · Balazs Scheidler
Quoted 2 lines · Patrick McHardy

I'm wondering if it would be possible to use normal input routing combined with netfilter marks to do the diversion ..

Quoted 8 lines

The problem is that userspace proxies open ports dynamically (think of FTP data channels), you cannot add iptables rule for every such redirection. So one rule for every dynamic redirection is a no-go.

If we'd add a single rule, which would do some kind of lookup and then mark packets, would again introduce a state inside tproxy that'd need to be synchronized with the socket table. We explicitly wanted to avoid such tables.

How exactly are dynamic ports handled? Do you just add a catch-all rule that filters based on socket lookups?

In that case you could do something like this:

ip route add local default dev lo scope host table 1 ip rule add fwmark 0x1 lookup 1

and still use the socket lookups for marking, which would (without the socket caching) remove the need for this patch entirely.

Quoted 2 lines

And additionally, using the mark this way would prevent the admin to use it they way he/she likes.

We support bitwise use of the mark everywhere in current kernels, so that shouldn't be a problem anymore.

#5 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

Quoted 2 lines · Patrick McHardy

We support bitwise use of the mark everywhere in current kernels, so that shouldn't be a problem anymore.

For firewall mark based policy routing to work, one must still disable rp_filter, because this lookup doesn't take the mark into account[1].

So this statement is not quite true, although I believe you are probably right for this case.

BTW: This rp_filter=0 requirement isn't even officially documented
(e.g. in the LARTC).

Regards

Ingo Oeser

[1] But does take TOS into account for historic (???) reasons.

#6 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

Quoted 8 lines · Ingo Oeser
Quoted 2 lines · Patrick McHardy

We support bitwise use of the mark everywhere in current kernels, so that shouldn't be a problem anymore.

For firewall mark based policy routing to work, one must still disable rp_filter, because this lookup doesn't take the mark into account[1].

If distributions wouldn't enable it by default there would be no need to disable it again :)

Quoted 5 lines

So this statement is not quite true, although I believe you are probably right for this case.

BTW: This rp_filter=0 requirement isn't even officially documented
(e.g. in the LARTC).

The kernel defaults to rp_filter = 0.

Quoted 1 line

[1] But does take TOS into account for historic (???) reasons.

With TOS the assumption of symetry is a bit more realistic than with fwmarks, but it will probably still not work properly if you actually use routing by TOS value.

#7 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

In reply to Patrick McHardy

Hi,

Quoted 10 lines · Patrick McHardy

How exactly are dynamic ports handled? Do you just add a catch-all rule that filters based on socket lookups?

In that case you could do something like this:

ip route add local default dev lo scope host table 1 ip rule add fwmark 0x1 lookup 1

and still use the socket lookups for marking, which would (without the socket caching) remove the need for this patch entirely.

Ok, I'll try to address all the concerns raised on the list.

Thanks a lot for the review and comments.

Signature · 2 lines
 Regards,
  Krisztian Kovacs
#8 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

In reply to Patrick McHardy

Hi,

Quoted 5 lines · Patrick McHardy
Quoted 5 lines
net/ipv4/route.c +5 -0
+	rcu_read_lock();
+	for (rth = rcu_dereference(rt_hash_table[hash].chain); rth;
+	     rth = rcu_dereference(rth->u.rt_next)) {
+		if (rth->fl.fl4_dst == iph->daddr &&
+		    rth->fl.fl4_src == iph->saddr &&
Quoted 8 lines

net/ipv4/route.c static int ip_rt_bug(struct sk_buff *skb · line 1563

Quoted 5 lines
net/ipv4/route.c +5 -0
+		    rth->fl.iif == iif &&
+		    rth->fl.oif == 0 &&
+		    rth->fl.mark == skb->mark &&
+		    (rth->u.dst.flags & DST_DIVERTED) &&
+		    rth->fl.fl4_tos == tos) {

Mark and tos look unnecessary here since they don't affect the further processing of the packet.

Indeed, thanks for spotting it.

Quoted 8 lines
Quoted 8 lines
net/ipv4/route.c +8 -0
+			rth->u.dst.lastuse = jiffies;
+			dst_hold(&rth->u.dst);
+			rth->u.dst.__use++;
+			RT_CACHE_STAT_INC(in_hit);
+			rcu_read_unlock();
+
+			dst_release(skb->dst);
+			skb->dst = (struct dst_entry*)rth;
Quoted 8 lines

net/ipv4/route.c static int ip_rt_bug(struct sk_buff *skb · line 1575

Quoted 4 lines
net/ipv4/route.c +4 -0
+
+			if (sk) {
+				sock_hold(sk);
+				skb->sk = sk;

This looks racy, the socket could be closed between the lookup and the actual use. Why do you need the socket lookup at all, can't you just divert all packets selected by iptables?

Yes, it's racy, but I this is true for the "regular" socket lookup, too.
Take UDP for example: __udp4_lib_rcv() does the socket lookup, gets a reference to the socket, and then calls udp_queue_rcv_skb() to queue the skb. As far as I can see there's nothing there which prevents the socket from being closed between these calls. sk_common_release() even documents this behaviour:

[...]
if (sk->sk_prot->destroy)
sk->sk_prot->destroy(sk);

/*
* Observation: when sock_common_release is called, processes have
* no access to socket. But net still has.
* Step one, detach it from networking:
*
* A. Remove from hash tables.
*/

sk->sk_prot->unhash(sk);

/*
* In this point socket cannot receive new packets, but it is possible
* that some packets are in flight because some CPU runs receiver and
* did hash table lookup before we unhashed socket. They will achieve
* receive queue and will be purged by socket destructor. *
* Also we still have packets pending on receive queue and probably,
* our own packets waiting in device queues. sock_destroy will drain
* receive queue, but transmitted packets will delay socket destruction
* until the last reference will be released.
*/
[...]

Of course it's true that doing early lookups and storing that reference
in the skb widens the window considerably, but I think this race is already handled. Or is there anything I don't see?

Signature · 2 lines
 Regards,
  Krisztian Kovacs
#9 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

Quoted 45 lines · KOVACS Krisztian
Quoted 7 lines · Patrick McHardy

net/ipv4/route.c static int ip_rt_bug(struct sk_buff *skb · line 1575

Quoted 3 lines
net/ipv4/route.c +3 -0
+			if (sk) {
+				sock_hold(sk);
+				skb->sk = sk;

This looks racy, the socket could be closed between the lookup and the actual use. Why do you need the socket lookup at all, can't you just divert all packets selected by iptables?

Yes, it's racy, but I this is true for the "regular" socket lookup, too.
Take UDP for example: __udp4_lib_rcv() does the socket lookup, gets a reference to the socket, and then calls udp_queue_rcv_skb() to queue the skb. As far as I can see there's nothing there which prevents the socket from being closed between these calls. sk_common_release() even documents this behaviour:

[...]
if (sk->sk_prot->destroy)
sk->sk_prot->destroy(sk);

/*
* Observation: when sock_common_release is called, processes have
* no access to socket. But net still has.
* Step one, detach it from networking:
*
* A. Remove from hash tables.
*/

sk->sk_prot->unhash(sk);

/*
* In this point socket cannot receive new packets, but it is possible
* that some packets are in flight because some CPU runs receiver and
* did hash table lookup before we unhashed socket. They will achieve
* receive queue and will be purged by socket destructor. *
* Also we still have packets pending on receive queue and probably,
* our own packets waiting in device queues. sock_destroy will drain
* receive queue, but transmitted packets will delay socket destruction
* until the last reference will be released.
*/
[...]

Quoted 3 lines

Of course it's true that doing early lookups and storing that reference
in the skb widens the window considerably, but I think this race is already handled. Or is there anything I don't see?

You're right, it seems to be handled properly (except I think there is a race between sk_common_release calling xfrm_sk_free_policy and f.e. udp calling __xfrm_policy_check, will look into that).

It probably shouldn't be cached anyway, with nf_queue for example the window could be _really_ large.

#10 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

Hi,

Quoted 10 lines · Patrick McHardy
Quoted 3 lines

Of course it's true that doing early lookups and storing that
reference in the skb widens the window considerably, but I think this race is already handled. Or is there anything I don't see?

You're right, it seems to be handled properly (except I think there is a race between sk_common_release calling xfrm_sk_free_policy and f.e. udp calling __xfrm_policy_check, will look into that).

It probably shouldn't be cached anyway, with nf_queue for example the window could be _really_ large.

Patrick, I seem to be out of ideas how this could be done
without "caching" the socket lookup. The problem is that it's not only caching in some cases. For example we can do something like this:

  iptables -t tproxy -A PREROUTING -s X -d Y -p tcp --dport 80 \
           -j TPROXY --to proxy_ip:proxy_port

In this case the TPROXY target does a socket lookup for
proxy_ip:proxy_port and stores that socket reference in skb->sk. Obviously if you don't do this then TCP will do a lookup on the packet's original destination address/port and it won't work.

Unfortunately I don't see any way how this could be solved without
storing the result of the lookup... So while I agree that having that socket reference in the skb is risky, as previously skb->sk was unused on the input path, I simply don't have any other idea. (Unless your load iptable_tproxy skb->sk==NULL on input is still true with these patches, so I think there should be absolutely no problems with tproxy unused.)

Other possible problems which came to my mind:

- The previous version was missing IPv4 fragment reassembly: we obviously need this to be able to do socket lookups, so now I've added this to iptable_tproxy.
- IP_FREEBIND does not require NET_ADMIN capability, combined with the relaxed source address on ip_output() this means that we provide a way to do IPv4 address forging for unprivileged users. As we must not break anything it looks like we need a separate socket option for disabling output source address checks (this would obviously require NET_ADMIN).

Thoughts? I'd be especially interested in any ideas wrt. the socket
reference problems, as the other two seems to be easier to solve.

Signature · 2 lines
 Regards,
  Krisztian Kovacs
#11 lore raw

Re: [PATCH/RFC 01/10] Implement local diversion of IPv4 skbs

Quoted 26 lines · KOVACS Krisztian
Quoted 10 lines · Patrick McHardy
Quoted 3 lines

Of course it's true that doing early lookups and storing that
reference in the skb widens the window considerably, but I think this race is already handled. Or is there anything I don't see?

You're right, it seems to be handled properly (except I think there is a race between sk_common_release calling xfrm_sk_free_policy and f.e. udp calling __xfrm_policy_check, will look into that).

It probably shouldn't be cached anyway, with nf_queue for example the window could be _really_ large.

Patrick, I seem to be out of ideas how this could be done
without "caching" the socket lookup. The problem is that it's not only caching in some cases. For example we can do something like this:

  iptables -t tproxy -A PREROUTING -s X -d Y -p tcp --dport 80 \
           -j TPROXY --to proxy_ip:proxy_port

In this case the TPROXY target does a socket lookup for
proxy_ip:proxy_port and stores that socket reference in skb->sk. Obviously if you don't do this then TCP will do a lookup on the packet's original destination address/port and it won't work.

Quoted 6 lines

Unfortunately I don't see any way how this could be solved without
storing the result of the lookup... So while I agree that having that socket reference in the skb is risky, as previously skb->sk was unused on the input path, I simply don't have any other idea. (Unless your load iptable_tproxy skb->sk==NULL on input is still true with these patches, so I think there should be absolutely no problems with tproxy unused.)

One (not very pretty) possibility would be to store the address/port somewhere in the skb and use it for the socket lookup. I think thats also what the 2.2 code did. Other than that I don't have any ideas either, but I'm not too familiar with that code, maybe someone else could explain whether caching the sockets would really be a problem and why.

Quoted 5 lines

Other possible problems which came to my mind:

- The previous version was missing IPv4 fragment reassembly: we obviously need this to be able to do socket lookups, so now I've added this to iptable_tproxy.

Makes sense.

Quoted 5 lines

- IP_FREEBIND does not require NET_ADMIN capability, combined with the relaxed source address on ip_output() this means that we provide a way to do IPv4 address forging for unprivileged users. As we must not break anything it looks like we need a separate socket option for disabling output source address checks (this would obviously require NET_ADMIN).

Also sounds reasonable.

#12 lore raw

PATCH 2/10 Port redirection support for TCP

In reply to KOVACS Krisztian

Current TCP code relies on the local port of the listening socket being the same as the destination address of the incoming connection. Port redirection used by many transparent proxying techniques obviously breaks this, so we have to store the original destination port address.

This patch extends struct inet_request_sock and stores the incoming destination port value there. It also modifies the handshake code to use that value as the source port when sending reply packets.

---

 include/net/inet_sock.h         |    1 +
 include/net/tcp.h               |    1 +
 net/ipv4/inet_connection_sock.c |    2 ++
 net/ipv4/tcp_output.c           |    2 +-
 4 files changed, 5 insertions(+), 1 deletions(-)
include/net/inet_sock.h +1 -0
index ce6da97..0bd167b 100644
@@ -64,6 +64,7 @@ #if defined(CONFIG_IPV6) || defined(CONF
 #endif
 	__be32			loc_addr;
 	__be32			rmt_addr;
+	__be16			loc_port;
 	__be16			rmt_port;
 	u16			snd_wscale : 4, 
 				rcv_wscale : 4, 
include/net/tcp.h +1 -0
index b7d8317..08ea8f3 100644
@@ -983,6 +983,7 @@ static inline void tcp_openreq_init(stru
 	ireq->acked = 0;
 	ireq->ecn_ok = 0;
 	ireq->rmt_port = skb->h.th->source;
+	ireq->loc_port = skb->h.th->dest;
 }
 
 extern void tcp_enter_memory_pressure(void);
net/ipv4/inet_connection_sock.c +2 -0
index 9d68837..889a487 100644
@@ -502,6 +502,8 @@ struct sock *inet_csk_clone(struct sock
 		newicsk->icsk_bind_hash = NULL;
 
 		inet_sk(newsk)->dport = inet_rsk(req)->rmt_port;
+		inet_sk(newsk)->num = ntohs(inet_rsk(req)->loc_port);
+		inet_sk(newsk)->sport = inet_rsk(req)->loc_port;
 		newsk->sk_write_space = sk_stream_write_space;
 
 		newicsk->icsk_retransmits = 0;
net/ipv4/tcp_output.c +1 -1
index 32c1a97..bb37048 100644
@@ -2132,7 +2132,7 @@ #endif
 	th->syn = 1;
 	th->ack = 1;
 	TCP_ECN_make_synack(req, th);
-	th->source = inet_sk(sk)->sport;
+	th->source = ireq->loc_port;
 	th->dest = ireq->rmt_port;
 	TCP_SKB_CB(skb)->seq = tcp_rsk(req)->snt_isn;
 	TCP_SKB_CB(skb)->end_seq = TCP_SKB_CB(skb)->seq + 1;
#13 lore raw

PATCH 3/10 Don't do the TCP socket lookup if we already have one attached

In reply to KOVACS Krisztian

TCP input code path looks up the TCP socket hash tables to find a socket matching the incoming packet. However, as iptable_tproxy does socket lookups early the skb may already have the appropriate reference attached, in that case we steal that reference instead of doing the lookup.

---

 net/ipv4/tcp_ipv4.c |   13 ++++++++++---
 1 files changed, 10 insertions(+), 3 deletions(-)
net/ipv4/tcp_ipv4.c +10 -3
index bf7a224..7828aec 100644
@@ -1647,9 +1647,16 @@ int tcp_v4_rcv(struct sk_buff *skb)
 	TCP_SKB_CB(skb)->flags	 = skb->nh.iph->tos;
 	TCP_SKB_CB(skb)->sacked	 = 0;
 
-	sk = __inet_lookup(&tcp_hashinfo, skb->nh.iph->saddr, th->source,
-			   skb->nh.iph->daddr, th->dest,
-			   inet_iif(skb));
+	if (unlikely(skb->sk)) {
+		/* steal reference */
+		sk = skb->sk;
+		skb->destructor = NULL;
+		skb->sk = NULL;
+	} else {
+		sk = __inet_lookup(&tcp_hashinfo, skb->nh.iph->saddr, th->source,
+				   skb->nh.iph->daddr, th->dest,
+				   inet_iif(skb));
+	}
 
 	if (!sk)
 		goto no_tcp_socket;
#14 lore raw

PATCH 4/10 Don't do the UDP socket lookup if we already have one attached

In reply to KOVACS Krisztian

UDP input code path looks up the UDP socket hash tables to find a socket matching the incoming packet. However, as iptable_tproxy does socket lookups early the skb may already have the appropriate reference attached, in that case we steal that reference instead of doing the lookup.

---

 net/ipv4/udp.c |   11 +++++++++--
 1 files changed, 9 insertions(+), 2 deletions(-)
net/ipv4/udp.c +9 -2
index cfff930..1b348f5 100644
@@ -1225,8 +1225,15 @@ int __udp4_lib_rcv(struct sk_buff *skb,
 	if(rt->rt_flags & (RTCF_BROADCAST|RTCF_MULTICAST))
 		return __udp4_lib_mcast_deliver(skb, uh, saddr, daddr, udptable);
 
-	sk = __udp4_lib_lookup(saddr, uh->source, daddr, uh->dest,
-			       skb->dev->ifindex, udptable        );
+	if (skb->sk) {
+		/* steal reference */
+		sk = skb->sk;
+		skb->destructor = NULL;
+		skb->sk = NULL;
+	} else {
+		sk = __udp4_lib_lookup(saddr, uh->source, daddr, uh->dest,
+				       skb->dev->ifindex, udptable        );
+	}
 
 	if (sk != NULL) {
 		int ret = udp_queue_rcv_skb(sk, skb);
#15 lore raw

PATCH 5/10 Remove local address check on IP output

In reply to KOVACS Krisztian

ip_route_output() contains a check to make sure that no flows with non-local source IP addresses are routed. Unfortunately this check makes it completely impossible to use non-local bound sockets as no outbound packets will make through the stack.

This patch moves the interface lookup to the multicast-specific code path as that is the only real user of the interface data looked up.

---

 net/ipv4/route.c |   13 +++++--------
 1 files changed, 5 insertions(+), 8 deletions(-)
net/ipv4/route.c +5 -8
index 537b976..bb1158a 100644
@@ -2498,11 +2498,6 @@ #endif
 		    ZERONET(oldflp->fl4_src))
 			goto out;
 
-		/* It is equivalent to inet_addr_type(saddr) == RTN_LOCAL */
-		dev_out = ip_dev_find(oldflp->fl4_src);
-		if (dev_out == NULL)
-			goto out;
-
 		/* I removed check for oif == dev_out->oif here.
 		   It was wrong for two reasons:
 		   1. ip_dev_find(saddr) can return wrong iface, if saddr is
@@ -2528,12 +2523,14 @@ #endif
 			   Luckily, this hack is good workaround.
 			 */
 
+			/* It is equivalent to inet_addr_type(saddr) == RTN_LOCAL */
+			dev_out = ip_dev_find(oldflp->fl4_src);
+			if (dev_out == NULL)
+				goto out;
+
 			fl.oif = dev_out->ifindex;
 			goto make_route;
 		}
-		if (dev_out)
-			dev_put(dev_out);
-		dev_out = NULL;
 	}
 
 
#16 lore raw

Re: [PATCH/RFC 05/10] Remove local address check on IP output

Quoted 20 lines · KOVACS Krisztian

ip_route_output() contains a check to make sure that no flows with non-local source IP addresses are routed. Unfortunately this check makes it completely impossible to use non-local bound sockets as no outbound packets will make through the stack.

This patch moves the interface lookup to the multicast-specific code path as that is the only real user of the interface data looked up.

---

 net/ipv4/route.c |   13 +++++--------
 1 files changed, 5 insertions(+), 8 deletions(-)
net/ipv4/route.c +0 -0
index 537b976..bb1158a 100644

net/ipv4/route.c #endif · line 2505

Quoted 8 lines
net/ipv4/route.c +0 -5
@@ -2498,11 +2498,6 @@ #endif
 		    ZERONET(oldflp->fl4_src))
 			goto out;
 
-		/* It is equivalent to inet_addr_type(saddr) == RTN_LOCAL */
-		dev_out = ip_dev_find(oldflp->fl4_src);
-		if (dev_out == NULL)
-			goto out;
-

I'm not sure how exactly this is used by applications, but couldn't you restrict this to sockets without freebind?

#17 lore raw

Re: [PATCH/RFC 05/10] Remove local address check on IP output

Hi,

Quoted 8 lines · Patrick McHardy
Quoted 8 lines
net/ipv4/route.c +0 -0
index 537b976..bb1158a 100644
@@ -2498,11 +2498,6 @@ #endif
 		    ZERONET(oldflp->fl4_src))
 			goto out;
 
Quoted 8 lines

net/ipv4/route.c #endif · line 2505

Quoted 5 lines
net/ipv4/route.c +0 -5
-		/* It is equivalent to inet_addr_type(saddr) == RTN_LOCAL */
-		dev_out = ip_dev_find(oldflp->fl4_src);
-		if (dev_out == NULL)
-			goto out;
-

I'm not sure how exactly this is used by applications, but couldn't you restrict this to sockets without freebind?

I'll try to do so in the next incarnation of the patches. Thanks for the
comment, it'd ineed be safer to do so.

BTW, could anyone shed some light on exactly why that check is
necessary? As far as I can see it prevents packets with a non-local source address being routed -- but I fail to see why we need to prevent that.

Signature · 2 lines
 Regards,
  Krisztian Kovacs
#18 lore raw

IP_FREEBIND and CAP_NET_ADMIN (was: Re: [PATCH/RFC 05/10] Remove local address check on IP output)

In reply to Patrick McHardy
Quoted 24 lines · Patrick McHardy
Quoted 23 lines · KOVACS Krisztian

ip_route_output() contains a check to make sure that no flows with non-local source IP addresses are routed. Unfortunately this check makes it completely impossible to use non-local bound sockets as no outbound packets will make through the stack.

This patch moves the interface lookup to the multicast-specific code path as that is the only real user of the interface data looked up.

---

 net/ipv4/route.c |   13 +++++--------
 1 files changed, 5 insertions(+), 8 deletions(-)
net/ipv4/route.c +0 -0
index 537b976..bb1158a 100644
@@ -2498,11 +2498,6 @@ #endif
 		    ZERONET(oldflp->fl4_src))
 			goto out;
 
Quoted 8 lines

net/ipv4/route.c #endif · line 2505

Quoted 5 lines
net/ipv4/route.c +0 -5
-		/* It is equivalent to inet_addr_type(saddr) == RTN_LOCAL */
-		dev_out = ip_dev_find(oldflp->fl4_src);
-		if (dev_out == NULL)
-			goto out;
-

I'm not sure how exactly this is used by applications, but couldn't you restrict this to sockets without freebind?

As it turned out since I've submitted this patch simply removing the branch in the quoted patch above is not good, as that'd allow all local users to generate connections from a non-local IP address. (Since setting IP_FREEBIND does not require CAP_NET_ADMIN.)

I've attempted to restrict the removal of the check to certain sockets, but it is more difficult than expected. It'd require touching a lot of areas of the kernel code, as the socket is not available at times where an output routing lookup is requested.

In fact the only thing available when making the decision in ip_route_output_slow() is a struct flowi. I've tried to stuff a flag bit into "struct flowi", but that solution seems to be very risky, as the value for "struct flowi->flags" is not consulted at a lot of places. IMHO the result would be far from pretty... (And I have to admit that I don't really know what flowi->flags is used for. I've found no in-tree user of that field. The only defined flag bit, FLOWI_FLAG_MULTIPATHOLDROUTE, has no in-tree user either.)

And even if we have this flag in place, it's not enough to set it for certain sockets in ip_route_connect(): this would not handle SYN+ACK or ACK packets sent in response for redirected TCP connection attempts. And who knows what else is still hiding there: ip_route_output_*() calls are pretty much everywhere in the whole net/ipv4 directory.

So I think the cleanest solution would be to require CAP_NET_ADMIN for IP_FREEBIND. This way, a non-root process would not be allowed to bind to a non-local socket, thus it would not be possible to initiate connections from a non-local IP.

As this would be a change in the kernel ABI, me and Balazs have tried to search for applications using the IP_FREEBIND option using Google codesearch (www.google.com/codesearch).

Outside libc and kernel, we've found only three applications that mention this option:
* socat: which allows setting all socket options by the user (I doubt using IP_FREEBIND with socat has any meaningful use)
* strace: to be able to dump IP_FREEBIND
* qemu: for emulating Linux system calls

Neither of these require IP_FREEBIND as core functionality, and will probably work if IP_FREEBIND would be bound to CAP_NET_ADMIN.

So the question is: shall we take the IP_FREEBIND approach, this would change a hardly ever used interface by requiring CAP_NET_ADMIN capabilities, or we should try finding all the scattered places in the Linux IP stack which does a route lookup?

Signature · 2 lines
 Regards,
  Krisztian Kovacs
maintainer
#19 lore raw

Re: IP_FREEBIND and CAP_NET_ADMIN

From: KOVACS Krisztian <[email protected]>
Date: Tue, 6 Feb 2007 15:36:18 +0100

Quoted 7 lines

Neither of these require IP_FREEBIND as core functionality, and will probably work if IP_FREEBIND would be bound to CAP_NET_ADMIN.

So the question is: shall we take the IP_FREEBIND approach, this would change a hardly ever used interface by requiring CAP_NET_ADMIN capabilities, or we should try finding all the scattered places in the Linux IP stack which does a route lookup?

We're not going to remove functionality from the user for the sake of convenience of something you are trying to write.

If it was some security hole, then fine, but it's not so it can stay and it does have legitimate uses.

This freebind behavior should actually be the default, but we had to put the socket option and sysctl there because allowing freebind by default makes several test suites fail that try to purposely bind to a non-local address and expect an error return.

It allows servers to bind when your on-demand connection is down.

#20 lore raw

PATCH 1/1 tg3 : avoid an expensive divide

Hi David

During an oprofile session of linux-2.6.20 on a dual opteron system, I noticed an expensive divide was done in tg3_poll().

I am using gcc-4.1.1, so the following comment from drivers/net/tg3.c seems over-optimistic :

/* Do not place this n-ring entries value into the tp struct itself,
* we really want to expose these constants to GCC so that modulo et
* al. operations are done with shifts and masks instead of with
* hw multiply/modulo instructions. Another solution would be to
* replace things like '% foo' with '& (foo - 1)'.
*/

#define TG3_RX_RCB_RING_SIZE(tp)        \
        ((tp->tg3_flags2 & TG3_FLG2_5705_PLUS) ?  512 : 1024)

Assembly code before patch :
(oprofile results included)

  6434  0.0088 :ffffffff803684b9:       mov    0x6f0(%r15),%eax
   587 8.0e-04 :ffffffff803684c0:       and    $0x40000,%eax
  2170  0.0030 :ffffffff803684c5:       cmp    $0x1,%eax
               :ffffffff803684c8:       lea    0x1(%r13),%eax
               :ffffffff803684cc:       sbb    %ecx,%ecx
  2051  0.0028 :ffffffff803684ce:       xor    %edx,%edx
               :ffffffff803684d0:       and    $0x200,%ecx
    20 2.7e-05 :ffffffff803684d6:       add    $0x200,%ecx
  1986  0.0027 :ffffffff803684dc:       div    %ecx
103427  0.1410 :ffffffff803684de:       cmp    %edx,0xffffffffffffff7c(%rbp)

Assembly code after the suggested patch :

ffffffff803684b9:           mov    0x6f0(%r15),%eax
ffffffff803684c0:           and    $0x40000,%eax
ffffffff803684c5:           cmp    $0x1,%eax
ffffffff803684c8:           sbb    %eax,%eax
ffffffff803684ca:           inc    %r13d
ffffffff803684cd:           and    $0x200,%eax
ffffffff803684d2:           add    $0x1ff,%eax
ffffffff803684d7:           and    %eax,%r13d
ffffffff803684da:           cmp    %r13d,0xffffffffffffff7c(%rbp)
drivers/net/tg3.c renamed from drivers/net/tg3.c.orig +1 -1
@@ -3384,7 +3384,7 @@
 		}
 next_pkt_nopost:
 		sw_idx++;
-		sw_idx %= TG3_RX_RCB_RING_SIZE(tp);
+		sw_idx &= (TG3_RX_RCB_RING_SIZE(tp) - 1);
 
 		/* Refresh hw_idx to see if there is new work */
 		if (sw_idx == hw_idx) {
maintainer
#21 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

From: Eric Dumazet <[email protected]>
Date: Tue, 06 Feb 2007 21:53:24 +0100

Quoted 11 lines

I am using gcc-4.1.1, so the following comment from drivers/net/tg3.c seems over-optimistic :

/* Do not place this n-ring entries value into the tp struct itself,
* we really want to expose these constants to GCC so that modulo et
* al. operations are done with shifts and masks instead of with
* hw multiply/modulo instructions. Another solution would be to
* replace things like '% foo' with '& (foo - 1)'.
*/

#define TG3_RX_RCB_RING_SIZE(tp)        \
         ((tp->tg3_flags2 & TG3_FLG2_5705_PLUS) ?  512 : 1024)

Because I've seen gcc optimize this properly before (at least on sparc64), it means that either:

1) There is a GCC bug where the properties of the constants do not propagate.

2) GCC really thinks the divide is cheaper (code density vs. cycle count tradeoffs etc.)

Anyways, we should probably apply your patch in any event.

Michael, any objections?

#22 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

Quoted 7 lines · David Miller

Because I've seen gcc optimize this properly before (at least on sparc64), it means that either:

1) There is a GCC bug where the properties of the constants do not propagate.

Quoted 7 lines

2) GCC really thinks the divide is cheaper (code density vs. cycle count tradeoffs etc.)

Anyways, we should probably apply your patch in any event.

Michael, any objections?

No objections. I will create another patch to convert similar divisions in other spots.

maintainer
#23 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

From: "Michael Chan" <[email protected]>
Date: Tue, 06 Feb 2007 14:09:16 -0800

Quoted 2 lines

No objections. I will create another patch to convert similar divisions in other spots.

I actually checked and those cases work on true constants so do not generate the divide instructions.

#24 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

In reply to David Miller
Quoted 9 lines · David Miller

Because I've seen gcc optimize this properly before (at least on sparc64), it means that either:

1) There is a GCC bug where the properties of the constants do not propagate.

2) GCC really thinks the divide is cheaper (code density vs. cycle count tradeoffs etc.)

Probably Eric compiled with the now default CONFIG_CC_OPTIMIZE_FOR_SIZE/-Os. With that gcc decides to use the shorter hardware divide instruction, even though it is significantly slower than an expanded optimized sequence for constant dividend.

We've seen this in a few other cases while during performance regression testing between kernels that still used -O2 vs the newer -Os.

No good solution found unfortunately.

-Andi

maintainer
#25 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

From: Andi Kleen <[email protected]>
Date: 07 Feb 2007 10:54:49 +0100

Quoted 9 lines

Probably Eric compiled with the now default CONFIG_CC_OPTIMIZE_FOR_SIZE/-Os. With that gcc decides to use the shorter hardware divide instruction, even though it is significantly slower than an expanded optimized sequence for constant dividend.

We've seen this in a few other cases while during performance regression testing between kernels that still used -O2 vs the newer -Os.

No good solution found unfortunately.

Thanks for the explanation, that makes a lot of sense.

#26 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

In reply to Andi Kleen
Quoted 15 lines · Andi Kleen
Quoted 8 lines · David Miller

Because I've seen gcc optimize this properly before (at least on sparc64), it means that either:

1) There is a GCC bug where the properties of the constants do not propagate.

2) GCC really thinks the divide is cheaper (code density vs. cycle count tradeoffs etc.)

Probably Eric compiled with the now default CONFIG_CC_OPTIMIZE_FOR_SIZE/-Os. With that gcc decides to use the shorter hardware divide instruction, even though it is significantly slower than an expanded optimized sequence for constant dividend.

Quoted 4 lines

We've seen this in a few other cases while during performance regression testing between kernels that still used -O2 vs the newer -Os.

No good solution found unfortunately.

Well, this could explain but unfortunatly I dont have this option set :

# grep OPTIMIZE .config
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
# gcc -v
Using built-in specs.
Target: x86_64-unknown-linux-gnu
Configured with: ../gcc-4.1.1/configure --enable-languages=c,c++
Thread model: posix
gcc version 4.1.1

Then I did :

# make drivers/net/tg3.s
  CHK     include/linux/version.h
  CHK     include/linux/utsrelease.h
  CC      drivers/net/tg3.s
# more drivers/net/tg3.s
        .file   "tg3.c"
# GNU C version 4.1.1 (x86_64-unknown-linux-gnu)
#       compiled by GNU C version 4.1.0.
# GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
# options passed:  -nostdinc -Iinclude -D__KERNEL__ -DCONFIG_AS_CFI=1
# -DKBUILD_STR(s)=#s -DKBUILD_BASENAME=KBUILD_STR(tg3)
# -DKBUILD_MODNAME=KBUILD_STR(tg3) -isystem -include -MD -march=k8 -m64
# -mno-red-zone -mcmodel=kernel -mno-sse -mno-mmx -mno-sse2 -mno-3dnow
# -maccumulate-outgoing-args -auxbase-strip -O2 -Wall -Wundef
# -Wstrict-prototypes -Wno-trigraphs -Wno-sign-compare
# -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-aliasing
# -fno-common -fno-reorder-blocks -fno-asynchronous-unwind-tables
# -funit-at-a-time -fno-omit-frame-pointer -fno-optimize-sibling-calls
# -fno-stack-protector -fverbose-asm
# options enabled:  -falign-loops -fargument-alias -fbranch-count-reg
# -fcaller-saves -fcprop-registers -fcrossjumping -fcse-follow-jumps
# -fcse-skip-blocks -fdefer-pop -fdelete-null-pointer-checks
# -fearly-inlining -feliminate-unused-debug-types -fexpensive-optimizations
# -ffunction-cse -fgcse -fgcse-lm -fguess-branch-probability -fident
# -fif-conversion -fif-conversion2 -finline-functions-called-once
# -fipa-pure-const -fipa-reference -fipa-type-escape -fivopts
# -fkeep-static-consts -fleading-underscore -floop-optimize
# -floop-optimize2 -fmath-errno -fmerge-constants -foptimize-register-move
# -fpeephole -fpeephole2 -freg-struct-return -fregmove -freorder-functions
# -frerun-cse-after-loop -frerun-loop-opt -fsched-interblock -fsched-spec
# -fsched-stalled-insns-dep -fschedule-insns2 -fshow-column
# -fsplit-ivs-in-unroller -fstrength-reduce -fthread-jumps -ftrapping-math
# -ftree-ccp -ftree-ch -ftree-copy-prop -ftree-copyrename -ftree-dce
# -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-loop-im
# -ftree-loop-ivcanon -ftree-loop-optimize -ftree-lrs -ftree-pre
# -ftree-salias -ftree-sink -ftree-sra -ftree-store-ccp
# -ftree-store-copy-prop -ftree-ter -ftree-vect-loop-version -ftree-vrp
# -funit-at-a-time -fverbose-asm -fzero-initialized-in-bss
# -m128bit-long-double -m64 -m80387 -maccumulate-outgoing-args
# -malign-stringops -mfancy-math-387 -mfp-ret-in-387 -mieee-fp
# -mno-red-zone -mpush-args -mtls-direct-seg-refs

# Compiler executable checksum: a068cb1f6a9c2e4d8616444230e91dfc

Eric

#27 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

Quoted 10 lines

Well, this could explain but unfortunatly I dont have this option set :

# grep OPTIMIZE .config
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
# gcc -v
Using built-in specs.
Target: x86_64-unknown-linux-gnu
Configured with: ../gcc-4.1.1/configure --enable-languages=c,c++

Ah. Looking more closely the problem is different (sorry for the wrong explanation earlier, it fit too well and it really happened elsewhere)

The code is

#define TG3_RX_RCB_RING_SIZE(tp)        \
        ((tp->tg3_flags2 & TG3_FLG2_5705_PLUS) ?  512 : 1024)

...
sw_idx %= TG3_RX_RCB_RING_SIZE(tp);

The problem is that gcc doesn't rewrite the x % (cond ? const1 : const2) expression to the more optimizable cond ? (x % const1 : x % const2). For the first it sees a full variable division, for which div* is the best code to generate.

Your patch is probably correct or alternatively rewrite the expression to the optimizable form by hand.

Arguably gcc should do this by itself. Perhaps file a gcc bug about this missing optimization.

-Andi

#28 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

In reply to Eric Dumazet

drivers/net/tg3.c line 3390

Quoted 13 lines · Eric Dumazet

plain text document attachment (tg3_avoid_divide.patch)

drivers/net/tg3.c renamed from drivers/net/tg3.c.orig +1 -1
@@ -3384,7 +3384,7 @@
 		}
 next_pkt_nopost:
 		sw_idx++;
-		sw_idx %= TG3_RX_RCB_RING_SIZE(tp);
+		sw_idx &= (TG3_RX_RCB_RING_SIZE(tp) - 1);
 
 		/* Refresh hw_idx to see if there is new work */
 		if (sw_idx == hw_idx) {

There are other places doing similar divide (e.g. tg3_recycle_rx(), tg3_alloc_rx_skb()). I think we should change them all.

maintainer
#29 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

From: "Michael Chan" <[email protected]>
Date: Tue, 06 Feb 2007 14:05:41 -0800

Quoted 13 lines · Eric Dumazet
Quoted 12 lines

plain text document attachment (tg3_avoid_divide.patch)

drivers/net/tg3.c renamed from drivers/net/tg3.c.orig +1 -1
@@ -3384,7 +3384,7 @@
 		}
 next_pkt_nopost:
 		sw_idx++;
-		sw_idx %= TG3_RX_RCB_RING_SIZE(tp);
+		sw_idx &= (TG3_RX_RCB_RING_SIZE(tp) - 1);
 
Quoted 5 lines

drivers/net/tg3.c line 3390

Quoted 2 lines
drivers/net/tg3.c renamed from drivers/net/tg3.c.orig +0 -0
 		/* Refresh hw_idx to see if there is new work */
 		if (sw_idx == hw_idx) {

There are other places doing similar divide (e.g. tg3_recycle_rx(), tg3_alloc_rx_skb()). I think we should change them all.

Ok, I'll take care of this.

#30 lore raw

Re: [PATCH] tg3 : avoid an expensive divide

In reply to Michael Chan
Quoted 13 lines · Michael Chan
Quoted 11 lines · Eric Dumazet

plain text document attachment (tg3_avoid_divide.patch)

drivers/net/tg3.c renamed from drivers/net/tg3.c.orig +1 -1
@@ -3384,7 +3384,7 @@
 		}
 next_pkt_nopost:
 		sw_idx++;
-		sw_idx %= TG3_RX_RCB_RING_SIZE(tp);
+		sw_idx &= (TG3_RX_RCB_RING_SIZE(tp) - 1);
 
Quoted 5 lines

drivers/net/tg3.c line 3390

Quoted 2 lines
drivers/net/tg3.c renamed from drivers/net/tg3.c.orig +0 -0
 		/* Refresh hw_idx to see if there is new work */
 		if (sw_idx == hw_idx) {

There are other places doing similar divide (e.g. tg3_recycle_rx(), tg3_alloc_rx_skb()). I think we should change them all.

Well, I checked whole tg3.o with objdump --disassemble and there are only three points using a div : tg3_nvram_phys_addr()/tg3_set_eeprom() & tg3_open(), wich are seldom used :)

Eric

#32 lore raw

PATCH 6/10 Create a tproxy flag in struct sk_buff

In reply to KOVACS Krisztian

We would like to be able to match on whether or not a given packet has been diverted by tproxy. To make this possible we need a flag in sk_buff.

---

 include/linux/skbuff.h |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
include/linux/skbuff.h +2 -1
index 4ff3940..6d7f5c7 100644
@@ -284,7 +284,8 @@ struct sk_buff {
 				nfctinfo:3;
 	__u8			pkt_type:3,
 				fclone:2,
-				ipvs_property:1;
+				ipvs_property:1,
+				ip_tproxy:1;
 	__be16			protocol;
 
 	void			(*destructor)(struct sk_buff *skb);
#33 lore raw

PATCH 7/10 Export UDP socket lookup function

In reply to KOVACS Krisztian

The iptables tproxy code has to be able to do UDP socket hash lookups, so we have to provide an exported lookup function for this purpose.

---

 include/net/udp.h |    4 ++++
 net/ipv4/udp.c    |    8 ++++++++
 2 files changed, 12 insertions(+), 0 deletions(-)
include/net/udp.h +4 -0
index 1b921fa..ea5aa31 100644
@@ -141,6 +141,10 @@ extern int 	udp_lib_setsockopt(struct so
 				   char __user *optval, int optlen,
 				   int (*push_pending_frames)(struct sock *));
 
+extern struct sock *udp4_lib_lookup(__be32 saddr, __be16 sport,
+				    __be32 daddr, __be16 dport,
+				    int dif);
+
 DECLARE_SNMP_STAT(struct udp_mib, udp_statistics);
 /*
  * 	SNMP statistics for UDP and UDP-Lite
net/ipv4/udp.c +8 -0
index 1b348f5..a44d3d3 100644
@@ -284,6 +284,14 @@ static struct sock *__udp4_lib_lookup(__
 	return result;
 }
 
+struct sock *udp4_lib_lookup(__be32 saddr, __be16 sport,
+			     __be32 daddr, __be16 dport,
+			     int dif)
+{
+	return __udp4_lib_lookup(saddr, sport, daddr, dport, dif, udp_hash);
+}
+EXPORT_SYMBOL_GPL(udp4_lib_lookup);
+
 static inline struct sock *udp_v4_mcast_next(struct sock *sk,
 					     __be16 loc_port, __be32 loc_addr,
 					     __be16 rmt_port, __be32 rmt_addr,
#34 lore raw

PATCH 8/10 iptables tproxy table

In reply to KOVACS Krisztian

The iptables tproxy table registers a new hook on PRE_ROUTING and for each incoming TCP/UDP packet performs as follows:

1. Does a TCP/UDP socket hash lookup to decide whether or not the packet is sent to a non-local bound socket. If a matching socket is found and the socket has the IP_FREEBIND socket option enabled the skb is diverted locally and the socket reference is stored in the skb.

2. If no matching socket was found, the PREROUTING chain of the iptables tproxy table is consulted. Matching rules with the TPROXY target can do transparent redirection here. (In this case it is not necessary to have the IP_FREEBIND socket option enabled for the target socket, redirection takes place even for "regular" sockets. This way no modification of the application is necessary.)

---

 include/linux/netfilter_ipv4/ip_tproxy.h |   20 ++
 net/ipv4/netfilter/Kconfig               |   10 +
 net/ipv4/netfilter/Makefile              |    1 
 net/ipv4/netfilter/iptable_tproxy.c      |  253 ++++++++++++++++++++++++++++++
 4 files changed, 284 insertions(+), 0 deletions(-)
include/linux/netfilter_ipv4/ip_tproxy.h new file +20 -0
new file mode 100644
index 0000000..ae890e3
@@ -0,0 +1,20 @@
+#ifndef _IP_TPROXY_H
+#define _IP_TPROXY_H
+
+#include <linux/types.h>
+
+/* look up and get a reference to a matching socket */
+extern struct sock *
+ip_tproxy_get_sock(const u8 protocol,
+		   const __be32 saddr, const __be32 daddr,
+		   const __be16 sport, const __be16 dport,
+		   const struct net_device *in);
+
+/* divert skb to a given socket */
+extern int
+ip_tproxy_do_divert(struct sk_buff *skb,
+		    const struct sock *sk,
+		    const int require_freebind,
+		    const struct net_device *in);
+
+#endif
net/ipv4/netfilter/Kconfig +10 -0
index f6026d4..312b0ef 100644
@@ -652,6 +652,16 @@ config IP_NF_RAW
 	  If you want to compile it as a module, say M here and read
 	  <file:Documentation/modules.txt>.  If unsure, say `N'.
 
+# tproxy table
+config IP_NF_TPROXY
+	tristate "Transparent proxying"
+	depends on IP_NF_IPTABLES
+	help
+	  Transparent proxying. For more information see
+	  http://www.balabit.com/downloads/tproxy.
+
+	  To compile it as a module, choose M here.  If unsure, say N.
+
 # ARP tables
 config IP_NF_ARPTABLES
 	tristate "ARP tables support"
net/ipv4/netfilter/Makefile +1 -0
index 15e741a..aa57ce4 100644
@@ -71,6 +71,7 @@ obj-$(CONFIG_IP_NF_MANGLE) += iptable_ma
 obj-$(CONFIG_IP_NF_NAT) += iptable_nat.o
 obj-$(CONFIG_NF_NAT) += iptable_nat.o
 obj-$(CONFIG_IP_NF_RAW) += iptable_raw.o
+obj-$(CONFIG_IP_NF_TPROXY) += iptable_tproxy.o
 
 # matches
 obj-$(CONFIG_IP_NF_MATCH_IPRANGE) += ipt_iprange.o
net/ipv4/netfilter/iptable_tproxy.c new file +253 -0

253 lines not on this page · raw message

#35 lore raw

Re: [PATCH/RFC 08/10] iptables tproxy table

Quoted 27 lines · KOVACS Krisztian
net/ipv4/netfilter/iptable_tproxy.c new file +21 -0
new file mode 100644
index 0000000..6049c83
@@ -0,0 +1,253 @@
+/*
+ * Transparent proxy support for Linux/iptables
+ *
+ * Copyright (c) 2006-2007 BalaBit IT Ltd.
+ * Author: Balazs Scheidler, Krisztian Kovacs
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/version.h>
+#include <linux/module.h>
+
+#include <linux/sysctl.h>
+#include <linux/vmalloc.h>
+#include <linux/net.h>
+#include <linux/slab.h>
+#include <linux/if.h>
+#include <linux/proc_fs.h>

net/ipv4/netfilter/iptable_tproxy.c line 31

Quoted 10 lines
net/ipv4/netfilter/iptable_tproxy.c new file +10 -0
+#include <linux/seq_file.h>
+#include <linux/netdevice.h>
+#include <linux/inetdevice.h>
+#include <linux/time.h>
+#include <linux/in.h>
+#include <net/tcp.h>
+#include <net/udp.h>
+#include <net/sock.h>
+#include <net/inet_sock.h>
+#include <asm/uaccess.h>

A few of these look unnecessary (like vmalloc, seq_file, proc_fs, time, uaccess).

Quoted 158 lines
net/ipv4/netfilter/iptable_tproxy.c new file +158 -0
+#include <linux/netfilter.h>
+#include <linux/netfilter_ipv4.h>
+#include <linux/netfilter_ipv4/ip_tables.h>
+
+#define TPROXY_VALID_HOOKS (1 << NF_IP_PRE_ROUTING)
+
+#if 0
+#define DEBUGP printk
+#else
+#define DEBUGP(f, args...)
+#endif
+
+static struct
+{
+	struct ipt_replace repl;
+	struct ipt_standard entries[2];
+	struct ipt_error term;
+} initial_table __initdata = {
+	.repl = {
+		.name = "tproxy",
+		.valid_hooks = TPROXY_VALID_HOOKS,
+		.num_entries = 2,
+		.size = sizeof(struct ipt_standard) + sizeof(struct ipt_error),
+		.hook_entry = {
+			[NF_IP_PRE_ROUTING] = 0 },
+		.underflow = {
+			[NF_IP_PRE_ROUTING] = 0 },
+	},
+	.entries = {
+		/* PRE_ROUTING */
+		{
+			.entry = {
+				.target_offset = sizeof(struct ipt_entry),
+				.next_offset = sizeof(struct ipt_standard),
+			},
+			.target = {
+				.target = {
+					.u = {
+						.target_size = IPT_ALIGN(sizeof(struct ipt_standard_target)),
+					},
+				},
+				.verdict = -NF_ACCEPT - 1,
+			},
+		},
+	},
+	/* ERROR */
+	.term = {
+		.entry = {
+			.target_offset = sizeof(struct ipt_entry),
+			.next_offset = sizeof(struct ipt_error),
+		},
+		.target = {
+			.target = {
+				.u = {
+					.user = {
+						.target_size = IPT_ALIGN(sizeof(struct ipt_error_target)),
+						.name = IPT_ERROR_TARGET,
+					},
+				},
+			},
+			.errorname = "ERROR",
+		},
+	}
+};
+
+static struct ipt_table tproxy_table = {
+	.name		= "tproxy",
+	.valid_hooks	= TPROXY_VALID_HOOKS,
+	.lock		= RW_LOCK_UNLOCKED,
+	.me		= THIS_MODULE,
+	.af		= AF_INET,
+};
+
+struct sock *
+ip_tproxy_get_sock(const u8 protocol,
+		   const __be32 saddr, const __be32 daddr,
+		   const __be16 sport, const __be16 dport,
+		   const struct net_device *in)
+{
+	struct sock *sk = NULL;
+
+	/* look up socket */
+	switch (protocol) {
+	case IPPROTO_TCP:
+		sk = __inet_lookup(&tcp_hashinfo,
+				   saddr, sport, daddr, sport,
+				   in->ifindex);
+		break;
+	case IPPROTO_UDP:
+		sk = udp4_lib_lookup(saddr, sport, daddr, dport,
+				     in->ifindex);
+		break;
+	default:
+		WARN_ON(1);
+	}
+
+	return sk;
+}
+EXPORT_SYMBOL_GPL(ip_tproxy_get_sock);
+
+int
+ip_tproxy_do_divert(struct sk_buff *skb, struct sock *sk,
+		    const int require_freebind,
+		    const struct net_device *in)
+{
+	const struct inet_sock *inet = inet_sk(sk);
+	struct in_device *indev;
+
+	if (unlikely(inet == NULL))
+		return -EINVAL;
+
+	if (!require_freebind || inet->freebind) {
+		indev = in_dev_get(in);
+		if (indev == NULL)
+			return -ENODEV;
+
+		skb->ip_tproxy = 1;
+
+		ip_divert_local(skb, indev, sk);
+		in_dev_put(indev);
+
+		DEBUGP(KERN_DEBUG "IP_TPROXY: diverted to socket %p\n", sk);
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ip_tproxy_do_divert);
+
+static unsigned int
+ip_tproxy_prerouting(unsigned int hooknum,
+		     struct sk_buff **pskb,
+		     const struct net_device *in,
+		     const struct net_device *out,
+		     int (*okfn)(struct sk_buff *))
+{
+	int verdict = NF_ACCEPT;
+	struct sk_buff *skb = *pskb;
+	u8 protocol = skb->nh.iph->protocol;
+	struct sock *sk = NULL;
+	const struct iphdr *iph = (*pskb)->nh.iph;
+	struct udphdr _hdr, *hp;
+
+	/* TCP and UDP only */
+	if ((protocol != IPPROTO_TCP) && (protocol != IPPROTO_UDP))
+		return NF_ACCEPT;
+
+	if (in == NULL)
+		return NF_ACCEPT;
+
+	if ((skb->dst != NULL) || (skb->ip_tproxy == 1))
+		return NF_ACCEPT;
+
+	hp = skb_header_pointer(skb, skb->nh.iph->ihl * 4, sizeof(_hdr), &_hdr);
+	if (hp == NULL) {
+		DEBUGP(KERN_DEBUG "IP_TPROXY: ip_tproxy_fn(): "
+		       "failed to get protocol header\n");
+		return NF_DROP;
+	}

net/ipv4/netfilter/iptable_tproxy.c line 200

Quoted 10 lines
net/ipv4/netfilter/iptable_tproxy.c new file +10 -0
+
+	sk = ip_tproxy_get_sock(iph->protocol,
+				iph->saddr, iph->daddr,
+				hp->source, hp->dest, in);
+	if (sk) {
+		if (ip_tproxy_do_divert(skb, sk, 1, in) < 0) {
+			DEBUGP(KERN_DEBUG "IP_TPROXY: divert failed, dropping packet\n");
+			verdict = NF_DROP;
+		}
+		sock_put(sk);

This doesn't handle time wait sockets (need inet_twsk_put).

net/ipv4/netfilter/iptable_tproxy.c line 213

Quoted 13 lines
net/ipv4/netfilter/iptable_tproxy.c new file +13 -0
+	} else {
+		verdict = ipt_do_table(pskb, hooknum, in, out, &tproxy_table);
+	}
+
+	return verdict;
+}
+
+static struct nf_hook_ops ip_tproxy_pre_ops = {
+	.hook		= ip_tproxy_prerouting,
+	.owner		= THIS_MODULE,
+	.pf		= PF_INET,
+	.hooknum	= NF_IP_PRE_ROUTING,
+	.priority	= -130

This should go in netfilter_ipv4.h

Quoted 41 lines
net/ipv4/netfilter/iptable_tproxy.c new file +40 -0
+};
+
+static int __init init(void)
+{
+	int ret;
+
+	ret = ipt_register_table(&tproxy_table, &initial_table.repl);
+	if (ret < 0) {
+		printk("IP_TPROXY: can't register tproxy table.\n");
+		return ret;
+	}
+
+	ret = nf_register_hook(&ip_tproxy_pre_ops);
+	if (ret < 0) {
+		printk("IP_TPROXY: can't register prerouting hook.\n");
+		goto clean_table;
+	}
+
+	printk("IP_TPROXY: Transparent proxy support initialized, version 4.0.0\n"
+	       "IP_TPROXY: Copyright (c) 2006-2007 BalaBit IT Ltd.\n");
+
+	return ret;
+
+ clean_table:
+	ipt_unregister_table(&tproxy_table);
+	return ret;
+}
+
+static void __exit fini(void)
+{
+	nf_unregister_hook(&ip_tproxy_pre_ops);
+	ipt_unregister_table(&tproxy_table);
+}
+
+module_init(init);
+module_exit(fini);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Krisztian Kovacs <[email protected]>");
+MODULE_DESCRIPTION("iptables transparent proxy table");
#36 lore raw

PATCH 9/10 iptables TPROXY target

In reply to KOVACS Krisztian

The TPROXY target implements redirection of non-local TCP/UDP traffic to local sockets. It is simply a wrapper around functionality exported from iptable_tproxy.

---

 include/linux/netfilter_ipv4/ipt_TPROXY.h |    9 +++
 net/ipv4/netfilter/Kconfig                |   11 +++
 net/ipv4/netfilter/Makefile               |    1 
 net/ipv4/netfilter/ipt_TPROXY.c           |  103 +++++++++++++++++++++++++++++
 4 files changed, 124 insertions(+), 0 deletions(-)
include/linux/netfilter_ipv4/ipt_TPROXY.h new file +9 -0
new file mode 100644
index 0000000..d05c956
@@ -0,0 +1,9 @@
+#ifndef _IPT_TPROXY_H_target
+#define _IPT_TPROXY_H_target
+
+struct ipt_tproxy_target_info {
+	u_int16_t lport;
+	u_int32_t laddr;
+};
+
+#endif
net/ipv4/netfilter/Kconfig +11 -0
index 312b0ef..7f76ab6 100644
@@ -662,6 +662,17 @@ config IP_NF_TPROXY
 
 	  To compile it as a module, choose M here.  If unsure, say N.
 
+config IP_NF_TARGET_TPROXY
+	tristate "TPROXY target support"
+	depends on IP_NF_TPROXY
+	help
+	  This option adds a `TPROXY' target, which is somewhat similar to
+	  REDIRECT.  It can only be used in the tproxy table and is useful
+	  to redirect traffic to a transparent proxy.  It does _not_ depend
+	  on Netfilter connection tracking.
+
+	  To compile it as a module, choose M here.  If unsure, say N.
+
 # ARP tables
 config IP_NF_ARPTABLES
 	tristate "ARP tables support"
net/ipv4/netfilter/Makefile +1 -0
index aa57ce4..851da93 100644
@@ -97,6 +97,7 @@ obj-$(CONFIG_IP_NF_TARGET_ULOG) += ipt_U
 obj-$(CONFIG_IP_NF_TARGET_TCPMSS) += ipt_TCPMSS.o
 obj-$(CONFIG_IP_NF_TARGET_CLUSTERIP) += ipt_CLUSTERIP.o
 obj-$(CONFIG_IP_NF_TARGET_TTL) += ipt_TTL.o
+obj-$(CONFIG_IP_NF_TARGET_TPROXY) += ipt_TPROXY.o
 
 # generic ARP tables
 obj-$(CONFIG_IP_NF_ARPTABLES) += arp_tables.o
net/ipv4/netfilter/ipt_TPROXY.c new file +103 -0

103 lines not on this page · raw message

#37 lore raw

Re: [PATCH/RFC 09/10] iptables TPROXY target

Quoted 32 lines · KOVACS Krisztian
net/ipv4/netfilter/ipt_TPROXY.c new file +26 -0
new file mode 100644
index 0000000..6f64717
 
+static unsigned int
+target(struct sk_buff **pskb,
+       const struct net_device *in,
+       const struct net_device *out,
+       unsigned int hooknum,
+       const struct xt_target *target,
+       const void *targinfo)
+{
+	const struct iphdr *iph = (*pskb)->nh.iph;
+	unsigned int verdict = NF_ACCEPT;
+	struct sk_buff *skb = *pskb;
+	struct udphdr _hdr, *hp;
+	struct sock *sk;
+
+	/* TCP/UDP only */
+	if ((iph->protocol != IPPROTO_TCP) &&
+	    (iph->protocol != IPPROTO_UDP))
+		return NF_ACCEPT;
+
+	if (in == NULL)
+		return NF_ACCEPT;
+
+	if ((skb->dst != NULL) || (skb->ip_tproxy == 1))
+		return NF_ACCEPT;
+
+	hp = skb_header_pointer(*pskb, iph->ihl * 4, sizeof(_hdr), &_hdr);

net/ipv4/netfilter/ipt_TPROXY.c line 59

Quoted 10 lines
net/ipv4/netfilter/ipt_TPROXY.c new file +10 -0
+	if (hp == NULL)
+		return NF_DROP;
+
+	sk = ip_tproxy_get_sock(iph->protocol,
+				iph->saddr, iph->daddr,
+				hp->source, hp->dest, in);
+	if (sk != NULL) {
+		if (ip_tproxy_do_divert(skb, sk, 0, in) < 0)
+			verdict = NF_DROP;
+		sock_put(sk);

Missing time wait socket handling.

Quoted 6 lines
net/ipv4/netfilter/ipt_TPROXY.c new file +6 -0
+	}
+
+	return verdict;
+}
+
+static int

net/ipv4/netfilter/ipt_TPROXY.c line 75

Quoted 10 lines
net/ipv4/netfilter/ipt_TPROXY.c new file +10 -0
+checkentry(const char *tablename,
+	   const void *e,
+	   const struct xt_target *target,
+           void *targinfo,
+           unsigned int hook_mask)
+{
+	/* checks are now done by the x_tables core based on
+	 * information specified in the ipt_target structure */
+	return 1;
+}

The function is optional, you can simply delete it.

Quoted 4 lines
net/ipv4/netfilter/ipt_TPROXY.c new file +4 -0
+
+static struct ipt_target ipt_tproxy_reg = {
+	.name		= "TPROXY",
+	.target		= target,

net/ipv4/netfilter/ipt_TPROXY.c line 89

Quoted 10 lines
net/ipv4/netfilter/ipt_TPROXY.c new file +10 -0
+	.targetsize	= sizeof(struct ipt_tproxy_target_info),
+	.table		= "tproxy",
+	.checkentry	= checkentry,
+	.me		= THIS_MODULE,
+};
+
+static int __init init(void)
+{
+	if (ipt_register_target(&ipt_tproxy_reg))
+		return -EINVAL;

This should return the result of ipt_register_target.

#38 lore raw

PATCH 10/10 iptables tproxy match

In reply to KOVACS Krisztian

Implements an iptables module which matches packets which have the tproxy flag set, that is, packets diverted in the tproxy table.

---

 net/netfilter/Kconfig     |    9 +++++
 net/netfilter/Makefile    |    1 +
 net/netfilter/xt_tproxy.c |   77 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 87 insertions(+), 0 deletions(-)
net/netfilter/Kconfig +9 -0
index 1b853c3..76c6f14 100644
@@ -559,6 +559,15 @@ config NETFILTER_XT_MATCH_QUOTA
 	  If you want to compile it as a module, say M here and read
 	  <file:Documentation/modules.txt>.  If unsure, say `N'.
 
+config NETFILTER_XT_MATCH_TPROXY
+	tristate '"tproxy" match support'
+	depends on NETFILTER_XTABLES
+	help
+	  This option adds a `tproxy' match, which allows you to match
+	  packets which have been diverted to local sockets by TProxy.
+
+	  To compile it as a module, choose M here.  If unsure, say N.
+
 config NETFILTER_XT_MATCH_REALM
 	tristate  '"realm" match support'
 	depends on NETFILTER_XTABLES
net/netfilter/Makefile +1 -0
index 5dc5574..4a83585 100644
@@ -62,6 +62,7 @@ obj-$(CONFIG_NETFILTER_XT_MATCH_MARK) +=
 obj-$(CONFIG_NETFILTER_XT_MATCH_MULTIPORT) += xt_multiport.o
 obj-$(CONFIG_NETFILTER_XT_MATCH_POLICY) += xt_policy.o
 obj-$(CONFIG_NETFILTER_XT_MATCH_PKTTYPE) += xt_pkttype.o
+obj-$(CONFIG_NETFILTER_XT_MATCH_TPROXY) += xt_tproxy.o
 obj-$(CONFIG_NETFILTER_XT_MATCH_QUOTA) += xt_quota.o
 obj-$(CONFIG_NETFILTER_XT_MATCH_REALM) += xt_realm.o
 obj-$(CONFIG_NETFILTER_XT_MATCH_SCTP) += xt_sctp.o
net/netfilter/xt_tproxy.c new file +77 -0

77 lines not on this page · raw message

#39 lore raw
In reply to KOVACS Krisztian
Quoted 10 lines · KOVACS Krisztian

The following set of patches implement transparent proxying support loosely modeled on the Linux 2.2 transparent proxying functionality.

In the last few years we've been maintaining a set of patches implementing Netfilter NAT to provide similar functionality. However, as time passed, more and more bugs surfaced, some of which were not possible to fix using that approach. Also, those patches required modification of user-space application code and the "API" provided was neither clean nor easy to use.

Quoted 6 lines

So instead of using NAT to dynamically redirect traffic to local addresses, we now rely on "native" non-locally-bound sockets and do early socket lookups for inbound IPv4 packets. These lookups are done in a separate Netfilter/iptables module, so there are only negligible performance implications of building transparent proxying support as a module and then not loading it.

Out of curiosity, would you use netchannels [1] if the implementation will be much broader? Since what you have created works exactly like netchannels netfilter NAT target (although it does not change ports, but it can be trivially extended), but without all existing netfilter overhead and without hacks in core TCP/UDP/IP/route code.

Some quote for netfilter maillist on behalf of advertisement :)

Network channel is peer-to-peer protocol agnostic communication channel between hardware and userspace. They allow to work directly with network hardware from userspace without any kind of filtering or processing from kernel.

Network channels are organized into single multidimensional trie, which allows to perform route, netfilter and other types of lookups in one traversal, since it is completely protocol agnostic.

Layering model does not exist in netchannels - layers are the way to design protocols, not implement them, thus actual protocol processing happens on the ends of netchannel - for example in userspace (userspace network stack), which improves cache locality and reduce overhead of unneded layer crossing.

Netchannels featureset includes:
* multidimensional wildcards support
* RCU searching
* single multidimensional trie for different kinds of dataflows
* dedicated processing threads with possibility to
schedule processing on different CPUs for those
netchannel types which are not acked with processing context
* userspace netchannel backend (allows to receive
packets to userspace), which can be used for:
o high-performance sniffers
o tun/tap device replacement
o packet socket replacement (note, that netchannels steal packets from main stack)
o userspace network stack implementation [2]
o own protocol stack implementaion (from VPN tunnels to TOE)
* netfilter netchannel backend (only NAT is supported as the most interesting user, NAT caches appropriate route, so essentially routing becomes part of the netchannel trie)

1. Netchannels homepage.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel

2. Userspace network stack.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=unetstack

3. Netchannel vs. socket benchmarks.
http://tservice.net.ru/~s0mbre/blog/2006/10/26#2006_10_26
http://tservice.net.ru/~s0mbre/blog/2006/12/21#2006_12_21

4. Netchannels multidimensional wildcard trie testing. userspace test (scales to millions of end nodes)
http://tservice.net.ru/~s0mbre/blog/2006/12/02#2006_12_02
kernelspace test (tens of thousands of netchannels)
http://tservice.net.ru/~s0mbre/blog/2006/12/21#2006_12_21_1

Signature · 1 line
	Evgeniy Polyakov
#40 lore raw

Hi Evgeniy,

Quoted 5 lines · Evgeniy Polyakov

Out of curiosity, would you use netchannels [1] if the implementation will be much broader? Since what you have created works exactly like netchannels netfilter NAT target (although it does not change ports, but it can be trivially extended), but without all existing netfilter overhead and without hacks in core TCP/UDP/IP/route code.

Indeed, a netchannels based implementation would be very nice. Combined
with a userspace network stack I think this could be a very powerful tool, especially for people doing dirty tricks -- like transparent proxying in our case.

However, I think that adopting netchannels now would be an enormous work
on our part. Of course, personally I'm really interested in netchannels and the related projects, but I agree with Harald that we still have a long way to go before being able to switch to netchannels. And I definitely _hate_ the previous incarnations of our tproxy patches enough that even this patchset seems acceptable for me. ;)

Signature · 2 lines
 Regards,
  Krisztian Kovacs
#41 lore raw
In reply to KOVACS Krisztian
Quoted 2 lines · KOVACS Krisztian

The following set of patches implement transparent proxying support loosely modeled on the Linux 2.2 transparent proxying functionality.

In a transparent http proxy server I wrote a while ago, we used to use tproxy for making outgoing connections appear to be originating from a foreign IP address, but moved to inserting an iptables nat rule from the proxy app every time an outgoing connection needs to be made, due to the pain of having to patch in the tproxy patches every time we needed to do a kernel update.

I'd love to see working tproxy functionality merged upstream for that reason alone.

I'd also love to see the old tproxy API go away entirely. It was always a bit of a pain to use.

#42 lore raw

Hi,

Quoted 2 lines · Lennert Buytenhek

I'd also love to see the old tproxy API go away entirely. It was always a bit of a pain to use.

It's gone with these patches: all you need is to bind() to foreign
addresses, like in the Linux 2.2 days.

Signature · 2 lines
 Regards,
  Krisztian Kovacs
#43 lore raw
Quoted 5 lines · KOVACS Krisztian
Quoted 2 lines

I'd also love to see the old tproxy API go away entirely. It was always a bit of a pain to use.

It's gone with these patches: all you need is to bind() to foreign
addresses, like in the Linux 2.2 days.

That's how I understood it. Great.

#44 lore raw
In reply to KOVACS Krisztian

Hi Krisztian!

Quoted 3 lines · KOVACS Krisztian

So instead of using NAT to dynamically redirect traffic to local addresses, we now rely on "native" non-locally-bound sockets and do early socket lookups for inbound IPv4 packets.

It's good to see a solid implementation of this 'old idea'.

Just as a quick historical note to netdev: This is the way how the netfilter project advised the balabit guys to implement fully transparent proxy support, after having seen the complexity of the old nat-based TPROXY patches.

So I personally support this patchset and vote for it to be included (with whatever modifications netdev deems apropriate)

It might be that there now is the experimental netchannels system which might provide an even better way for transparent proxy support.

However, ever since ip_tables was merged in the 2.3.x days, we have lacked good support for transparent proxies. Now that the first incarnation of the NAT based TPROXY patch for 2.4.x had to be developed and maintained out-of-tree for many years, I definitely think it's better to merge the new, way less intrusive, patchset.

Some interested party can work on a netchannels implementation later on, but that's the next generation...

Cheers,

Signature · 5 lines
- Harald Welte <[email protected]>                 http://netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie
#45 lore raw
Quoted 10 lines · Harald Welte
Quoted 3 lines

So instead of using NAT to dynamically redirect traffic to local addresses, we now rely on "native" non-locally-bound sockets and do early socket lookups for inbound IPv4 packets.

It's good to see a solid implementation of this 'old idea'.

Just as a quick historical note to netdev: This is the way how the netfilter project advised the balabit guys to implement fully transparent proxy support, after having seen the complexity of the old nat-based TPROXY patches.

Didn't rusty tell the balabit guys to use the NAT approach?

#46 lore raw
Quoted 6 lines · Lennert Buytenhek
Quoted 4 lines · Harald Welte
Quoted 3 lines

So instead of using NAT to dynamically redirect traffic to local addresses, we now rely on "native" non-locally-bound sockets and do early socket lookups for inbound IPv4 packets.

Quoted 8 lines
Quoted 6 lines

It's good to see a solid implementation of this 'old idea'.

Just as a quick historical note to netdev: This is the way how the netfilter project advised the balabit guys to implement fully transparent proxy support, after having seen the complexity of the old nat-based TPROXY patches.

Didn't rusty tell the balabit guys to use the NAT approach?

that was originally, way back. It turned out to be a bad idea, after all... way too complex. At least that's how I look at it. Too sad :(

Rusty and me then had the idea about the routing based approach at some point, if I remember correctly. We talked about it with Krisztian and Balazs at least on one occasion.

All that isn't really important. All I wanted to say was:

"I (and AFAIR the netfilter core team) believe this is the way to implement good support for transparent proxying. It's already the second completely independent implementation, let's merge it after all."

Signature · 5 lines
- Harald Welte <[email protected]>                 http://netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie