Question on advanced routing and/or virtual routers.

Hello!

I am curious if a single machine can be made to look like several routers. Please consider the following configuration:

Linux Router-A has 4 ethernet interfaces. PC-A is connected to eth0 and has IP 192.168.0.2. eth0 on Router-A has IP 192.168.0.1/24
eth1 has IP 192.168.1.1/24, and eth1 is connected directly to eth2
eth2 has IP 192.168.2.1/24
eth3 is connected 'upstream' and has IP 192.168.3.1/24

I would like for PC-A to be able to ping 192.168.0.1 and 192.168.1.1 as normal.
The part I'm not sure how to make work is that I want to be able to ping 192.168.2.1 and
have the packet route out of eth1 and into eth2 (PC-A -> eth0 -> eth1 -> eth2), and have the return packet follow the eth2 -> eth1 -> eth0 -> PC-A path. A trace-route from PC-A should show each of these hops (or, at least eth0 and eth2.)

The eventual goal is to have arbitrary numbers of 'routers' in a single Linux machine for emulation purposes.

I was thinking that I might could accomplish this using multiple routing tables and perhaps specific subnet routes for each each virtual router, specifying which interface the packets should leave in order to find the next hop.

Has anyone tried something similar to this or have ideas for how to best proceed?

Thanks,
Ben

Ben Greear <[email protected]> Candela Technologies Inc  http://www.candelatech.com

I started something like this a while ago (posted at <[email protected]> with a couple of replies by Thomas Graf, but I can't seem to find it in the archives) but then dropped the ball. It seems to work fairly well with a one-line kernel patch to allow route lookup before the local address check. Oh, and I didn't get traceroute working quite right either - I think there was some trick to finding the source address for the generated reply.

I've got some info and code here: http://dsg.cs.ubc.ca/~brendan/remus/

Thanks for the pointers!

I took a look at this, and I think I'm starting to understand it. But, I am not sure why the patch is needed. I was thinking that something like this should cause packets entering a particular interface to use a particular routing table. However, this does not seem to work for me (without the patch). I have yet to try with the patch.

I'm using my re-direct device patch, which is a pair of network devices that act like two ethernet interfaces connected with a cross-over cable. So, when you tx on A, the pkt is RX'd on B.

I am trying to set up two 'routers':

router 1001:

 rddA2:  10.0.3.1/24  -- connects to rddA1, which is in router 2
 rddC1:  10.0.4.1/24  -- connects to rest of world...unused currently

router 1002:

 rddA1:  10.0.3.2/24  -- connects to rddA2 in router 1
 rddB1:  10.0.2.1/24  -- connects to private network...unused currently

I want any packets received on the rddA1 and rddB1 interfaces to use routing table 1002, and those received on rddA2 and rddC1 to use routing table 1001.

It appears that the ping-response packets are using the local table, so matching by incoming device does not appear to be working as I expected. The ping requests are going out the right interface, so I think that matching on source IP addr is working.

My ip route related commands:

ip ru del from 10.0.3.1 lookup 1001
ip ru del from 0/0 lookup 1001
ip ru del from 10.0.4.1 lookup 1001
ip ru del from 0/0 lookup 1001
ip ru del from 10.0.2.1 lookup 1002
ip ru del from 0/0 lookup 1002
ip ru del from 10.0.3.2 lookup 1002
ip ru del from 0/0 lookup 1002
ip link set rddC1 down
ip link set rddC1 up
ip addr flush dev rddC1
ip addr add 10.0.4.1/24 broadcast 10.0.4.255 dev rddC1 ip rule add iif rddC1 lookup 1001
ip rule add from 10.0.4.1/32 table 1001
ip route add 10.0.4.0/24 via 10.0.4.2 table 1001
ip link set rddA2 down
ip link set rddA2 up
ip addr flush dev rddA2
ip addr add 10.0.3.1/24 broadcast 10.0.3.255 dev rddA2 ip rule add iif rddA2 lookup 1001
ip rule add from 10.0.3.1/32 table 1001
ip route add 10.0.3.0/24 via 10.0.3.2 table 1001
ip link set rddA1 down
ip link set rddA1 up
ip addr flush dev rddA1
ip addr add 10.0.3.2/24 broadcast 10.0.3.255 dev rddA1 ip rule add iif rddA1 lookup 1002
ip rule add from 10.0.3.2/32 table 1002
ip route add 10.0.3.0/24 via 10.0.3.1 table 1002
ip link set rddB1 down
ip link set rddB1 up
ip addr flush dev rddB1
ip addr add 10.0.2.1/24 broadcast 10.0.2.255 dev rddB1 ip rule add iif rddB1 lookup 1002
ip rule add from 10.0.2.1/32 table 1002
ip route add 10.0.2.0/24 via 10.0.2.2 table 1002 [root@lanforge-33-1E ~]# ping -I 10.0.3.1 10.0.3.2 PING 10.0.3.2 (10.0.3.2) from 10.0.3.1 : 56(84) bytes of data. 64 bytes from 10.0.3.2: icmp_seq=1 ttl=64 time=0.079 ms 64 bytes from 10.0.3.2: icmp_seq=2 ttl=64 time=0.062 ms 64 bytes from 10.0.3.2: icmp_seq=3 ttl=64 time=0.065 ms 64 bytes from 10.0.3.2: icmp_seq=4 ttl=64 time=0.062 ms

rddA2 Link encap:Ethernet HWaddr 00:19:DC:3A:39:50
inet addr:10.0.3.1 Bcast:10.0.3.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1 errors:0 dropped:0 overruns:0 frame:0 TX packets:45 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0
RX bytes:42 (42.0 b) TX bytes:4186 (4.0 KiB)

[root@lanforge-33-1E ~]# ifconfig rddA1
rddA1 Link encap:Ethernet HWaddr 00:62:D5:12:AF:31
inet addr:10.0.3.2 Bcast:10.0.3.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:45 errors:0 dropped:0 overruns:0 frame:0 TX packets:1 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0
RX bytes:4186 (4.0 KiB) TX bytes:42 (42.0 b)

For other's benefit, here is the patch suggested by Mr. Cully:

@@ -94,6 +94,7 @@
 static struct fib_rule local_rule = {
        .r_next =       &main_rule,
        .r_clntref =    ATOMIC_INIT(2),
+       .r_preference = 0x100,
        .r_table =      RT_TABLE_LOCAL,
        .r_action =     RTN_UNICAST,
 };

Thanks,
Ben

Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com

Using your scripts as a starting point, and the one-liner kernel patch, I was able to get it mostly working as well. I also see trouble with traceroute and agree that the problem is getting the right source-address in the ICMP and/or UDP response packets.

I'm going to start digging into the kernel to see if there is some way I can force the response out the same interface as it came in from, and also to force the source IP to be the same as the port it is leaving on.

Thanks,
Ben

Ben Greear <[email protected]> Candela Technologies Inc  http://www.candelatech.com