REDIRECT problem

Hello All,

For some time now, I've been using REDIRECT (roughly) like this:

$IPT -t nat -A PREROUTING -j proxyt
$IPT -t nat -A proxyt -d <network> -p tcp -m tcp -k RETURN
(repeated lots of times for different networks)
$IPT -t nat -A proxyt -p tcp -m tcp ! --dport 3128 -j REDIRECT --to-ports 3128

That works just fine: TCP connections to any network other than the specific get directed to the process listening on port 3128 (which then uses HTTP CONNECT on a proxy to connect to the Big Bad Internet).

I use ebtables on a pair of machines running Xen to redirect traffic to this machine:

    +-------------+     +-------------+
    | xen         |     |        xen  |
    |       +---+ |     | +---+       |
    | +---+ | P |=========| P'| +---+ |
    | | A | +-+-+ |     | +---+ | B | |
    | +---+   |   |     |       +---+ |
    |         |   |     |             |
    +---------|---+     +-------------+
              |

Big Bad Internet

Traffic from A is redirected (by ebtables) to P which has these rules on it and that connection is just fine. Traffic from B is directed to P' which then forwards traffic to P over a private network and until recently that worked just fine.

Previously, P was running Fedora 11 with the 2.6.30.10-105.2.16.fc11 kernel; but now its running Fedora13 with the 2.6.33.6-147.fc13.

So, previously on B I could connect to (say) google.com:80 and traffic was redirect to the process listing on port 3128 via P' and the private link and everything was fine.

Now, unfortunately, the same connection from B is hits the REDIRECT rule but the process listening on port 3128 doesn't come out of the accept(2) syscall. The same connection from A does work. The only visible difference is that traffic from A appears to come into P from eth0 and traffic from B appears to come from eth1. Inserting a LOG target immediately before the REDIRECT rule shows the packet hitting that REDIRECT (and one immediately after doesn't show anything so the REDIRECT is definitely matching).

Something seems to have changed between 2.6.30 and 2.6.33 and I'm at a loss to know what. I've looked around a bit, but so far haven't found anything.

Hopefully someone listening will be able to say "oh, you need to do <some magic>" :-) Or that this should never have worked in the first place because of something horrible I was relying on.

jch

In case anyone else comes looking this way, the problem commit was this one:

commit c1cf8422f0512c2b14f0d66bce34abb0645c888a

Author: Stephen Hemminger <[email protected]>
Date:   Fri Feb 20 08:25:36 2009 +0000

ip: add loose reverse path filtering

Extend existing reverse path filter option to allow strict or loose filtering. (See http://en.wikipedia.org/wiki/Reverse_path_filtering).

For compatibility with existing usage, the value 1 is chosen for
strict mode
and 2 for loose mode.

The routing issue that I was having was that packets coming in on the private link, having been redirected by ebtables on the second machine, would be routed out on the normal connection (not shown in the diagram) and so they we're being dropped before the REDIRECT rule even saw them.

The required magic was to set net.ipv4.conf.default.rp_filter=2

I suspect that this is going to bite quite a few people upgrading from RHEL5 to RHEL6 who have previously been relying on loose reverse path filtering with their asymmetric routing.

jch