NAT Port Forward problem in a not so simple network

Hello to all,

I have a NAT/Forward problem and I'm unable to solve, any help will be greatly appreciated.

I have a service that runs on a server that has no public IP, another server with the public IP should forward the port to it.

Scenario:

/--------\
|A:Cisco1|
\--------/
.
............
. .

/--------\  /--------\
|B:Cisco2|  |C:HPpro1|
\--------/  \--------/

.
.
/--------\
|D:Works1|
\--------/

Cisco1: Router where I can change nothing
Cisco2: Router where I can make very little changes
HPpro1: Server with Linux (Debian) and IPTables where I can change all *HELP HERE*
Works1: Workstation with a service listening on port 8080 TCP

A: Cisco1:
It has several public IPs and it redirect 1 IP on HPpro (all ports and protocols) LAN interface has 192.168.0.1, all works well here.
It has a port forward cisco rule that works (tested) it is:
ip nat inside source static 192.168.0.11 $PUBLICIP extendable (it is like NETMAP)

B: Cisco2:
It is used to let Workstation surf on the net
WAN interface has 192.168.0.2
LAN interface has 192.168.1.1, workstation are 192.168.1.2-20
It has a port forward cisco rule that works (tested) it is:
ip nat inside source static tcp 192.168.1.2 8080 192.168.0.2 8080 extendable

C: HPpro1: *HELP HERE*
It is a production server with many services on it
I would like to use its IP to test some application on a workstation instead of the production server
ETH0: 192.168.0.11
I need that it forward every packet that comes on 192.168.0.11:8080 (from 192.168.0.1) to 192.168.0.2

D: Works1
It has a service listening on port 8080
ETH0: 192.168.1.2

Lets say that everything works fine if I try to connect to
192.168.0.2:8080 (from 192.168.0.6 for example) I access to 192.168.1.2:8080 without problem

I need to access even connecting to 192.168.0.11:8080 (that is the IP with where if forwarded the public IP)

My try was this:
I create a simple port forward like this
DNAT tcp -- 0.0.0.0/0 192.168.0.11 tcp spts:1024:65535 dpt:41080 flags:0x17/0x02 state NEW to:192.168.0.2

It does not work, I sniffed all the packets on the hub/switch and I see that:
#, Source IP, Source MAC, Destination IP, Destionation MAC, Protocol, Info
1, 192.168.0.1, Cisco1, 192.168.0.11, HPpro1, TCP, 1234 > 8080 [SYN] Seq=0 Len=0 MSS=1460 2, 192.168.0.1, HPpro1, 192.168.0.02, Cisco2, TCP, 1234 > 8080 [SYN] Seq=0 Len=0 MSS=1460 3, 192.168.0.2, Cisco2, 192.168.0.01, Cisco1, TCP, 8080 > 1234 [SYN, ACK] Seq=780620 Ack=1 Len=0 MSS=1460 4, 192.168.0.1, Cisco1, 192.168.0.02, Cisco2, TCP, 1234 > 8080 [RST] Seq=1 Len=0

Packet 1 is the SYN request from Cisco1 to HPpro1
Packet 2 is strange, HPpro1 forwards to Cisco2 using the IP of Cisco1
Packet 3 Cisco2 does his job and the give the reply to Cisco1
Packet 4 Cisco1 reset the communication, perhaps because he wait a reply from HPpro1 and not Cisco2?

Sorry for long message but it was to better explain the problem

Many thanks for you help

Fabio De Paolis ha scritto:

Of course.
A possible solution is to SNAT packets exiting from 192.168.0.11 with destination 192.168.0.2.

*nod*

However, SNATing will have the possibly undesirable side effect of making the traffic appear to be from the machine that is doing the SNATing.

Grant. . . .

Hello,

It does not matter so much as, according to the packet trace, Cisco1 appears to SNAT incoming connections forwarded to HPpro1 anyway. So all you get in the logs is HPpro1's address instead of Cisco1's...

Huh! Great! Friggign lovely!

Do you know if that the prescribed Cisco way of doing that or just the way that it was configured at your site?

Seeing this new evidence, just DNAT & SNAT the traffic on the HP ProLiant and call it a day.

Grant. . . .

Let's break this down in to the simple network flow before trying to make things work (namely to make sure I understand what you are wanting to do).

Let's do a re-draw with some clarifications.

       +---+
       | Z |
       +-+-+
         |
         :       (INet)
         |
       +-+-+
=======| A |============
       +-+-+
         |
     +---+---+   (DMZ)
     |       |
   +-+-+   +-+-+
===| B |===| C |========
   +-+-+   +---+
     |           (LAN)
   +-+-+
   | D |
   +---+

A = Cisco 1
B = Cisco 2
C = HP ProLiant 1
D = Workstation 1
Z = INet client

Note: References to "External" (e) interfaces will be facing up and "Internal" (i) interfaces will be facing down.

I'm presuming (for the sake of discussion) that you presently you have globally routable services that appear to be on one or more Ae addresses that are port forwarded to Ce (and possibly others).

Now it sounds like you are wanting to port forward (one or more of) said service(s) from Ce to De with out changing any thing on A. So the traffic flow would be from client Z to Ae to Ce to De and back in the same direction.

I'm also guessing that you would like all systems to have the capability (whether or not you use it or firewall it is up to you) to connect to Ce and have the connection be port forwarded to De. Correct?

Before I go in to how to do this, please let me know if I'm on track or not.

*nod*

Grant. . . .

Grant Taylor ha scritto:

Absoluttely CORRECT, your description is very very good.

Another goal should be to minimize traffic on C for service running on D.

*nod* Now I know that I am on track and that it is safe to go down the path that I was thinking about.

Hum. This new goal may be problematic. The problem is that A is DNATing traffic to C that you now want to be re-directed elsewhere. So with out re-configuring A, the traffic is going to continue to be DNATed to C. What is better in the long run is to have A DNAT the traffic to B which will then DNAT the traffic in to D.

How much control do you have over B?

Can you request changes be made to A on your behalf?

I recently helped someone else on this list with a similar scenario. However in their scenario both C and D were directly connected to the internet via different providers and there was a VPN between C and D. The goal was to port forward connections originally to C over to D and have the replies go back through C and out to the original client. We ended up getting things to work exactly as they needed to. However all the traffic for the forwarded service was still passing through C on its way to D, which you are now wanting to avoid.

Grant. . . .

Grant Taylor ha scritto:

I have total control on B, even if fewer changes is good.

A is black box, it is from my service provider, I can change nothing, also request will be discarded at 99%

Yes on my knowledge I know that it can't be done without doubling the traffic on the net. I was wondering if at yuor knowledge the was another way. Of course if I could nat a port from A to B it would be easy and the traffic will me at minimum, but it cant be done.
I was wondering if there was a way to use C only for initial handshake and not for all packets, but it seems no.

Actually I'm with this iptables rules

iptables -nL -t nat

PREROUTING
DNAT tcp -- 0.0.0.0/0 192.168.0.11 tcp spts:1024:65535 dpts:8080 flags:0x17/0x02 state NEW to:192.168.0.2

POSTROUTING
SNAT tcp -- 0.0.0.0/0 192.168.0.2 tcp spts:1024:65535 dpts:8080 flags:0x17/0x02 state NEW to:192.168.0.11

It seems to work but this is the traffic I see on the net for a single packet #, Source IP(Source MAC), Destination IP(Destionation MAC), Protocol, Info 1, 192.168.0.01(Cisco1), 192.168.0.11(HPpro1), TCP, 1234 > 8080 [SYN] 2, 192.168.0.11(HPpro1), 192.168.0.02(Cisco2), TCP, 1234 > 8080 [SYN] 3, 192.168.0.02(Cisco2), 192.168.0.11(Cisco2), TCP, 8080 > 1234 [SYN, ACK] 4, 192.168.0.11(HPpro1), 192.168.0.01(Cisco1), TCP, 8080 > 1234 [SYN, ACK] and so on...

This is technically a Bounce. Let me know if this setup is correct, thanks!

Understood.

*sigh* I really dislike incompetent / uncooperative service providers.
You know that I had to ask.

Strictly speaking, on layer 3, no there is not any thing (that I am aware of) that can be done. However if you are willing to go down to in between layers 2 and 3 or even down to layer 2 there might be something that can be done.

         +---+
         | Z |
         +-+-+
           |
           :       (INet)
           |
         +-+-+
=========| A |============
         +-+-+
           |
   +---+---+---+   (DMZ)
   |   |       |
   | +-+-+   +-+-+
===|=| B |===| C |========
   | +-+-+   +---+
   |   |           (LAN)
   | +-+-+
   +-+ D |
     +---+

I'm guessing that there are other services on C that prevent you from moving it's IP to B. Correct?

I'm not sure how well this will work out (read: I don't know how well the Cisco will play in this game...) but you might be able to establish some sort of tunnel based forwarding from C to D so that inbound requests pass through the tunnel and replies go directly from D back out via A to the client.

Let's say for the sake of discussion that you add a connection from D back in to the DMZ (as above) and have this interface configured to *NOT* respond to ARP requests. If you do this, you could have the same IP bound to C as well as the new DMZ facing interface on D. With this type of set up, you could tunnel traffic from C to D via B and have D reply directly back with out passing through B or C.

In short, this is using the IP Tunnel mode of Linux Virtual Server to turn C in to a director for the single node back end. As such, your client Z would connect to Ae which is port forwarded to Ce which is tunnel to D which processes and replies to the client from the same IP as Ce. This means that A will send traffic to the IP that is bound to Ce and get replies from the same IP only bound to D's DMZ interface. The only difference that A should see is a different MAC address as the source for the reply traffic. However, if you spoof the MAC address, this will not be a problem. If you do spoof the MAC address you will need to do something like GARP to make sure the DMZ switch does not ""learn that the location of the shared MAC address is where D's DMZ interface is connected.

*nod*

As you can see, this will get the traffic where you want it to go, but it is far from the original traffic. Namely your source IP now looks like C, thus foiling any thing in logs on D.

I had not thought of what this might be called, but you are correct in that it will work with caveats.

Chew on the LVS idea and let me know if you want to run with it.

Grant. . . .

Grant Taylor ha scritto:

Thank you for the lesson.

D is too far from C's switch *sad*
and I think I can't add another cable

*nod*

*nod*

Well at least you were able to get things to work, all be it sub-optimal. Working is better than not.

Grant. . . .