custom routing (two gateways)
Hi all,
i'm having a somewhat stupid problem I can't get rid of.
we've a server that accepts incoming world connections
from a load balancer (10.10.10.4) to port 80,
and we still want to serve incoming ssh/http from the firewall (10.10.10.1) routed to this host (10.10.10.90) and their reply packets of cause shall be send out through the firewall.
unfortunately, both hosts (the load balancer (LB) and the firewall(FW)) are on the same subnet (10.10.10.0/24) and though on the same interface (eth0), but I need to find a solution.
So, packets send from the LB shall get their answer through the LB as nexthop of couse, as well as incoming packets from the FW shall have response packets send out to the FW as nexthop, too.
But how to realise this?
server: 10.10.10.90 (this is the problem host) firewall(FW): 10.10.10.1 (we receive (mostly) ssh/https conns from it) loadbalancer(LB): 10.10.10.4 (we receive http conns from it)
FW and LB are accepting/forwarding and routing connections from the world to our server.
server ~ # ip route list
192.168.0.0/24 dev eth1 proto kernel scope link src 192.168.0.90 10.10.10.0/24 dev eth0 proto kernel scope link src 10.10.10.90 127.0.0.0/8 dev lo scope link
default via 10.10.10.1 dev eth0
You see, default traffic is routed through the firewall as the default gateway... but now, we want to have outgoing traffic caused from incoming packets from the load balancer to be routed back through the loadbalancer itself.
I tried here several approaches, like adding custom routing tables, and modifying the tables (including main) either I got no answers routed to the FW or no traffic got routed to the LB.
Can you please give me a hint on how to find the right way?
Thanks in advance,
Christian Parpart.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 3 lines · Christian Parpart
I tried here several approaches, like adding custom routing tables, and modifying the tables (including main) either I got no answers routed to the FW or no traffic got routed to the LB.
I think you were on the right path.
I would set up a custom routing table for traffic associated with the load balancer to use. This load balancer routing table would use the load balancer as the default gateway.
You would then use ip rule(s) to determine which traffic would deviate from the normal default routing tables and use the load balancer routing table. This could probably be done based on source port on the web server, or based on connection / packet marks in IPTables. However you do it, you will probably need an additional routing table.
Keep going the direction you were, or perhaps post some of what you did try and let us take a look at it to see if you were close.
Grant. . . .
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 11 lines · On Monday 09 July 2007 21:01:26 Grant Taylor wrote:
Quoted 3 lines · Christian Parpart
I tried here several approaches, like adding custom routing tables, and modifying the tables (including main) either I got no answers routed to the FW or no traffic got routed to the LB.
I think you were on the right path.
I would set up a custom routing table for traffic associated with the load balancer to use. This load balancer routing table would use the load balancer as the default gateway.
Quoted 8 lines
You would then use ip rule(s) to determine which traffic would deviate from the normal default routing tables and use the load balancer routing table. This could probably be done based on source port on the web server, or based on connection / packet marks in IPTables. However you do it, you will probably need an additional routing table.
Keep going the direction you were, or perhaps post some of what you did try and let us take a look at it to see if you were close.
I finally found a way, and your hint (select by server port number) finally helped me to get rid of it :)
The following is the script to be executed at bootup to setup the additional routing table.
1 #! /bin/bash
2 # sets up additional routing table for load balancer traffic on a node 3
4
# --------------------------------------------------------------------------------
5 LB_IP=10.10.10.4 # load balancer IP
6 LB_IF=eth0 # ethernet interface the load balancer is
talking from/to
7
8 rt_table_name=loadbalancer # LB routing table name
9 rt_table_num=200 # LB routing table ID
10
11 fwmark=1 # FW mark to use for LB traffic
12
13 service_port=8000 # HTTP port for lighttpd on local mashine that
14 # serves for the load balancer
15
16
# -------------------------------------------------------------------------------- 17
18 # just ensure that we have a routing table called loadbalancer
19 if ! grep -q ${rt_table_name} /etc/iproute2/rt_tables; then
20 echo "${rt_table_num} ${rt_table_name}" >> /etc/iproute2/rt_tables
21 fi
22
23 # add a default route for communication from LB<->this_host
24 ip route flush table ${rt_table_name}
25 ip route add default via ${LB_IP} table ${rt_table_name} dev ${LB_IF} 26
27 # add a selector rule for which packets we want to use the LB routing table 28 ip rule add fwmark ${fwmark} table ${rt_table_name}
29
30 # finally lets mark all packets that shall be send out to the LB 31 iptables -t mangle -A OUTPUT -p tcp --sport ${service_port} -j MARK --set-mark ${fwmark}
32 # --(doesn't work? why?)-- iptables -t mangle -A INPUT -p tcp --dport ${service_port} -j CONNMARK --set-mark ${fwmark}
However, you might see, that I first tried to fwmark all packets by connection matching,
using CONNMARK. so that I only need to select all incoming traffic that came from the load balancer
as the previous hop and with our service port 8000, to let mark iptables itself all further connection
related packets automatically. but this didn't work out, unfortunately, and I am using the OUTPUT table
to match the packets.
What is the better approach anyway?
Thanks for your help,
Christian Parpart.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc