Skip to contentSkip to conversation

multiple nics, same isp, each nic independently

LARTC [email protected] 2 participants 3 messages May 25, 2003 – May 26, 2003

lore
Share & tools
lore
#0 lore raw

Hi, having some trouble with my routing, im kindof new to linux networking so my problem is probably easy to solve but i cant do it...

I have a quad nic connected to a switch, wich is connected to my isp, i get 3 ip:s by dhcp.

I want each of the nics to use there own gateway.. even if it’s the same on all nic:s... this is because I have bound different services to the different nics so I can get accurate transfer stats for every nic, separate ftp traffic from www and mail.

Eth0 = mail (public ip)
Eth1 = www (public ip)
Eth2 = ftp (public ip)
Eth3 = internal network (private ip)

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
222.144.190.0 0.0.0.0 255.255.255.128 U 0 0 0
eth0
222.144.190.0 0.0.0.0 255.255.255.128 U 0 0 0
eth2
222.144.190.0 0.0.0.0 255.255.255.128 U 0 0 0
eth1
192.168.150.0 0.0.0.0 255.255.255.0 U 0 0 0
eth3
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0
lo
0.0.0.0 222.144.190.1 0.0.0.0 UG 0 0 0
eth0

if I connect to ftp, eth2 incomming traffic goes in on eth2, ftp data (files and such) goes out eth0.

If I surf to my webserver eth1 incomming requests goes on eth1 but the actuall webpage is sent on eth0.

This is not what I want, I want both ftp-data and ftp-command to go in and out on eth2, www and the actual pages to go in and out on eth1. and mail to go in and out on eth0. so I want all the nics to use there own gateway. But I don’t know how to do it...

the solution to this is probably trivial but i have tryed all i could find info on... read the howto on lartc but the closest thing i can find is

4.2. Routing for multiple uplinks/providers

but that doesnt really apply to my "problem" since i want to use the same provider, but each nic independently...

would appriciate any help!
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#1 lore raw

Hi Johnny,

: I want each of the nics to use there own gateway.. even if it’s the : same on all nic:s... this is because I have bound different services to : the different nics so I can get accurate transfer stats for every nic, : separate ftp traffic from www and mail.

Interesting....might I ask why using netfilter (with connection tracking) doesn't give you the sort of accounting you need? I suspect that this would be much simpler than the solution you propose (restricting certain kinds of traffic to specific interfaces).

: Eth0 = mail (public ip)
: Eth1 = www (public ip)
: Eth2 = ftp (public ip)
: Eth3 = internal network (private ip)
:
: Kernel IP routing table

 : Destination     Gateway         Genmask         Flags Metric Ref    Use  Iface
 : 222.144.190.0   0.0.0.0         255.255.255.128 U     0      0        0  eth0
 : 222.144.190.0   0.0.0.0         255.255.255.128 U     0      0        0  eth2
 : 222.144.190.0   0.0.0.0         255.255.255.128 U     0      0        0  eth1
 : 192.168.150.0   0.0.0.0         255.255.255.0   U     0      0        0  eth3
 : 127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0  lo
 : 0.0.0.0         222.144.190.1   0.0.0.0         UG    0      0        0  eth0

Here's your problem. You have only one routing table. The main routing table. In order to solve this problem in the manner you describe, you'll need to use policy routing. You should become familiar with the concept of the RPDB and multiple routing tables before you can solve this problem.

: if I connect to ftp, eth2 incoming traffic goes in on eth2, ftp data : (files and such) goes out eth0. If I surf to my webserver eth1 : incomming requests goes on eth1 but the actuall webpage is sent on : eth0.
:
: This is not what I want, I want both ftp-data and ftp-command to go in and : out on eth2, www and the actual pages to go in and out on eth1. and mail : to go in and out on eth0. so I want all the nics to use there own gateway. : But I don’t know how to do it...

I expect that it would be easier for you to famaliarize yourself with iptables and use the accounting rules than it would be for you to become familiar with policy routing for such a simple problem.

: the solution to this is probably trivial but i have tryed all i could : find info on... read the howto on lartc but the closest thing i can : find is
:
: 4.2. Routing for multiple uplinks/providers
:
: but that doesnt really apply to my "problem" since i want to use the : same provider, but each nic independently...

I wouldn't recommend this solution--but as you note, this was the closest thing you could find. This is not to say that you couldn't specify output interface based on source IP. If you really wish to attempt this solution, write back to the list.

I'd suggest using a simpler technique like this:

iptables -t filter -I OUTPUT -s $DHCP0 -j ACCEPT iptables -t filter -I OUTPUT -s $DHCP1 -j ACCEPT iptables -t filter -I OUTPUT -s $DHCP2 -j ACCEPT

  iptables -t filter -I INPUT  -s $DHCP0 -j ACCEPT
  iptables -t filter -I INPUT  -s $DHCP1 -j ACCEPT
  iptables -t filter -I INPUT  -s $DHCP2 -j ACCEPT

Then, when you need to calculate how much bandwidth was used on each service, you can use iptable -nvL INPUT to see the total amount of bandwidth used in each of these rules.

If you are serious about calculating your bandwidth usage, you may find an IP accounting tool is well-suited for your needs. Consider something like iptraf [1] in background mode [2] or ipac-ng [3] (I assume you are running a 2.4 kernel).

Best of luck,

-Martin

 [1]  http://iptraf.seul.org/
 [2]  http://iptraf.seul.org/2.7/backop.html
 [3]  http://sourceforge.net/projects/ipac-ng/
      http://www.linux.org/apps/AppId_7462.html
Signature · 5 lines
Martin A. Brown --- SecurePipe, Inc. --- [email protected]

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#2 lore raw
In reply to [email protected]

Hi, and thank you for your reply. I read thru the script i had made with the routes and realized i had screwed up... i had forgotten to add a "dev eth2" when setting a default route on the table table_eth2.

# ip route add $net dev eth2 src $ip2 table table_eth2
# ip route add default via 213.114.191.1 -->dev eth2<-- table table_eth2
# ip rule add from $ip2 table table_eth2

so when adding the routes the last time i actually didnt change any route, but just adding the default route on eth0 in a different table... but now it all works fine.
Since the howto was for routing between 2 different isp:s it didnt have to apply what nic the rule was for, and since im kindof new to this i didnt see it either...
but now it all works as it should (i hope).

Thank you Martin.

Quoted 97 lines

Hi Johnny,

: I want each of the nics to use there own gateway.. even if it’s the : same on all nic:s... this is because I have bound different services to : the different nics so I can get accurate transfer stats for every nic, : separate ftp traffic from www and mail.

Interesting....might I ask why using netfilter (with connection tracking) doesn't give you the sort of accounting you need? I suspect that this would be much simpler than the solution you propose (restricting certain kinds of traffic to specific interfaces).

: Eth0 = mail (public ip)
: Eth1 = www (public ip)
: Eth2 = ftp (public ip)
: Eth3 = internal network (private ip)
:
: Kernel IP routing table
: Destination Gateway Genmask Flags Metric Ref Use
Iface
: 222.144.190.0 0.0.0.0 255.255.255.128 U 0 0 0
eth0
: 222.144.190.0 0.0.0.0 255.255.255.128 U 0 0 0
eth2
: 222.144.190.0 0.0.0.0 255.255.255.128 U 0 0 0
eth1
: 192.168.150.0 0.0.0.0 255.255.255.0 U 0 0 0
eth3
: 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0
lo
: 0.0.0.0 222.144.190.1 0.0.0.0 UG 0 0 0
eth0

Here's your problem. You have only one routing table. The main routing table. In order to solve this problem in the manner you describe, you'll need
to use policy routing. You should become familiar with the concept of the RPDB and multiple routing tables before you can solve this problem.

: if I connect to ftp, eth2 incoming traffic goes in on eth2, ftp data : (files and such) goes out eth0. If I surf to my webserver eth1 : incomming requests goes on eth1 but the actuall webpage is sent on : eth0.
:
: This is not what I want, I want both ftp-data and ftp-command to go in
and
: out on eth2, www and the actual pages to go in and out on eth1. and
mail
: to go in and out on eth0. so I want all the nics to use there own
gateway.
: But I don’t know how to do it...

I expect that it would be easier for you to famaliarize yourself with iptables and use the accounting rules than it would be for you to become familiar with policy routing for such a simple problem.

: the solution to this is probably trivial but i have tryed all i could : find info on... read the howto on lartc but the closest thing i can : find is
:
: 4.2. Routing for multiple uplinks/providers
:
: but that doesnt really apply to my "problem" since i want to use the : same provider, but each nic independently...

I wouldn't recommend this solution--but as you note, this was the closest thing you could find. This is not to say that you couldn't specify output interface based on source IP. If you really wish to attempt this solution, write back to the list.

I'd suggest using a simpler technique like this:

iptables -t filter -I OUTPUT -s $DHCP0 -j ACCEPT iptables -t filter -I OUTPUT -s $DHCP1 -j ACCEPT iptables -t filter -I OUTPUT -s $DHCP2 -j ACCEPT

  iptables -t filter -I INPUT  -s $DHCP0 -j ACCEPT
  iptables -t filter -I INPUT  -s $DHCP1 -j ACCEPT
  iptables -t filter -I INPUT  -s $DHCP2 -j ACCEPT

Then, when you need to calculate how much bandwidth was used on each service, you can use iptable -nvL INPUT to see the total amount of bandwidth used in each of these rules.

If you are serious about calculating your bandwidth usage, you may find an IP accounting tool is well-suited for your needs. Consider something like iptraf [1] in background mode [2] or ipac-ng [3] (I assume you are running a 2.4 kernel).

Best of luck,

-Martin

 [1]  http://iptraf.seul.org/
 [2]  http://iptraf.seul.org/2.7/backop.html
 [3]  http://sourceforge.net/projects/ipac-ng/
      http://www.linux.org/apps/AppId_7462.html
Quoted 7 lines

--
Martin A. Brown --- SecurePipe, Inc. --- [email protected]

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/