Networkconfiguration with KVM

Hi,

I am new to this list and to KVM (and qemu) so please be gentle with me. Up until now I have been running my virtualizing  using VMWare-Server. Now I want to try KVM due to some issues with the  VMWare-Server and I am having some troubles with the networking part of KVM.

This is a small example of what I want (best viewed in a fix-font):

 +-----------------------------------+
 | Host                              |
 |  +----------+                eth0 |---- 192.168.1.0/24
 |  |      eth0|-- +                 |
 |  | VM1  eth1|---(---+------- eth1 |---- 192.168.2.0/24
 |  |      eth2|---(---(---+         |
 |  +----------+   |   |   |         |
 |                 |   |   |         |
 |  +----------+   +---(---(--- eth2 |---- 192.168.1.0/24
 |  |      eth0|---+   |   |         |
 |  | VM2  eth1|-------+   +--- eth3 |---- 192.168.3.0/24
 |  |      eth2|-----------+         |
 |  +----------+                     |
 |                                   |
 +-----------------------------------+

Host-eth0 is only for the Host (no VM)
Host-eth1 is shared between the Host and the VM's (VM?-eth1) Host-eth2 and Host-eth3 are only for the VMs (eth0 and eth2)

The Host and the VMs all have fixed IPs (no dhcp or likewise). In this example th IPs could be:
Host-eth0:      192.168.1.1
Host-eth1:      192.168.2.1
Host-eth2:      -
Host-eth3:      -
VM1-eth0:               192.168.1.11
VM1-eth1:               192.168.2.11
VM1-eth2:               192.168.3.11
VM2-eth0:               192.168.1.22
VM2-eth1:               192.168.2.22
VM3-eth2:               192.168.3.22

And, yes, Host-eth0 and Host-eth2 are in the same subnet, with eth0 dedicated to the Host and eth2 dedicated to the VMs.

In VMWare this was quite easy to setup (three bridged networks).

Does someone know how I can set this up with KVM/QEMU?

Regards,
--
Dan Johansson, <http://www.dmj.nu>
***************************************************
This message is printed on 100% recycled electrons!
***************************************************
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Quoted 42 lines · Dan Johansson

Hi,

I am new to this list and to KVM (and qemu) so please be gentle with me. Up until now I have been running my virtualizing using VMWare-Server. Now I want to try KVM due to some issues with the VMWare-Server and I am having some troubles with the networking part of KVM.

This is a small example of what I want (best viewed in a fix-font):

 +-----------------------------------+
 | Host                              |
 |  +----------+                eth0 |---- 192.168.1.0/24
 |  |      eth0|-- +                 |
 |  | VM1  eth1|---(---+------- eth1 |---- 192.168.2.0/24
 |  |      eth2|---(---(---+         |
 |  +----------+   |   |   |         |
 |                 |   |   |         |
 |  +----------+   +---(---(--- eth2 |---- 192.168.1.0/24
 |  |      eth0|---+   |   |         |
 |  | VM2  eth1|-------+   +--- eth3 |---- 192.168.3.0/24
 |  |      eth2|-----------+         |
 |  +----------+                     |
 |                                   |
 +-----------------------------------+

Host-eth0 is only for the Host (no VM)
Host-eth1 is shared between the Host and the VM's (VM?-eth1) Host-eth2 and Host-eth3 are only for the VMs (eth0 and eth2)

The Host and the VMs all have fixed IPs (no dhcp or likewise). In this example th IPs could be:

Host-eth0:      192.168.1.1
Host-eth1:      192.168.2.1
Host-eth2:      -
Host-eth3:      -
VM1-eth0:               192.168.1.11
VM1-eth1:               192.168.2.11
VM1-eth2:               192.168.3.11
VM2-eth0:               192.168.1.22
VM2-eth1:               192.168.2.22
VM3-eth2:               192.168.3.22

Quoted 4 lines

And, yes, Host-eth0 and Host-eth2 are in the same subnet, with eth0 dedicated to the Host and eth2 dedicated to the VMs.

In VMWare this was quite easy to setup (three bridged networks).

Its easy with KVM too. You want 3 NICs per VM, so you need to pass the corresponding parameters(including qemu-ifup script) for 3 NICs to each VM.
In the host you need to create 2 bridges: say br-eth1 and br-eth2. Make them as the interface on the host in place of the corresponding eth interfaces.(brct addbr br-eth1; ifcfg eth1 0.0.0.0 up; brctl addif br-eth eth1; assign eth1's ip and routes to breth1; same for eth2). In the corresponding qemu-ifup scripts of each interface use bridge=br-ethN (This basicaly translates to brctl addif br-ethN $1, where $ is the tap device created)
This should work perfectly fine with your existing NW setup. For a quick reference use: http://www.linux-kvm.org/page/Networking

Quoted 48 lines · Dan Johansson

Quoted 42 lines

Hi,

I am new to this list and to KVM (and qemu) so please be gentle with me. Up until now I have been running my virtualizing using VMWare-Server. Now I want to try KVM due to some issues with the VMWare-Server and I am having some troubles with the networking part of KVM.

This is a small example of what I want (best viewed in a fix-font):

 +-----------------------------------+
 | Host                              |
 |  +----------+                eth0 |---- 192.168.1.0/24
 |  |      eth0|-- +                 |
 |  | VM1  eth1|---(---+------- eth1 |---- 192.168.2.0/24
 |  |      eth2|---(---(---+         |
 |  +----------+   |   |   |         |
 |                 |   |   |         |
 |  +----------+   +---(---(--- eth2 |---- 192.168.1.0/24
 |  |      eth0|---+   |   |         |
 |  | VM2  eth1|-------+   +--- eth3 |---- 192.168.3.0/24
 |  |      eth2|-----------+         |
 |  +----------+                     |
 |                                   |
 +-----------------------------------+

Host-eth0 is only for the Host (no VM)
Host-eth1 is shared between the Host and the VM's (VM?-eth1) Host-eth2 and Host-eth3 are only for the VMs (eth0 and eth2)

The Host and the VMs all have fixed IPs (no dhcp or likewise). In this example th IPs could be:

Host-eth0:      192.168.1.1
Host-eth1:      192.168.2.1
Host-eth2:      -
Host-eth3:      -
VM1-eth0:               192.168.1.11
VM1-eth1:               192.168.2.11
VM1-eth2:               192.168.3.11
VM2-eth0:               192.168.1.22
VM2-eth1:               192.168.2.22
VM3-eth2:               192.168.3.22

Quoted 4 lines

And, yes, Host-eth0 and Host-eth2 are in the same subnet, with eth0 dedicated to the Host and eth2 dedicated to the VMs.

In VMWare this was quite easy to setup (three bridged networks).

Quoted 12 lines

Its easy with KVM too. You want 3 NICs per VM, so you need to pass the corresponding parameters(including qemu-ifup script) for 3 NICs to each VM.
In the host you need to create 2 bridges: say br-eth1 and br-eth2. Make them as the interface on the host in place of the corresponding eth interfaces.(brct addbr br-eth1; ifcfg eth1 0.0.0.0 up; brctl addif br-eth eth1; assign eth1's ip and routes to breth1; same for eth2). In the corresponding qemu-ifup scripts of each interface use bridge=br-ethN (This basicaly translates to brctl addif br-ethN $1, where $ is the tap device created)
This should work perfectly fine with your existing NW setup. For a quick reference use: http://www.linux-kvm.org/page/Networking

Thanks for your help, but... I am still not able to get it to work the way I want.
This is what I have don so far:
brctl addbr br-eth1
brctl addbr br-eth3

ip link set eth1 up
ip link set eth3 up

brctl addif br-eth1 eth1
brctl addif br-eth3 eth3

tunctl -b -t qtap1
tunctl -b -t qtap3

brctl addif br-eth1 qtap1
brctl addif br-eth3 qtap3

ifconfig qtap1 up 0.0.0.0 promisc
ifconfig qtap3 up 0.0.0.0 promisc

# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0d:88:52:51:24
          inet addr:192.168.1.3  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:443638 errors:0 dropped:0 overruns:0 frame:0
          TX packets:758540 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:47041686 (44.8 MiB)  TX bytes:990115354 (944.2 MiB)
          Interrupt:19 Base address:0xec00

eth1 Link encap:Ethernet HWaddr 00:0d:88:52:51:25
inet addr:192.168.4.1 Bcast:192.168.4.255 Mask:255.255.255.0 UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:6 collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:360 (360.0 B) Interrupt:18 Base address:0xe880

eth3 Link encap:Ethernet HWaddr 00:0d:88:52:51:27
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:240 (240.0 B) Interrupt:16 Base address:0xe480

qtap1 Link encap:Ethernet HWaddr 26:c0:de:df:c5:e4
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:351 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500
RX bytes:14742 (14.3 KiB) TX bytes:0 (0.0 B)

qtap3 Link encap:Ethernet HWaddr 26:3e:ba:2d:97:bc
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500
RX bytes:252 (252.0 B) TX bytes:0 (0.0 B)

# brctl show
bridge name     bridge id               STP enabled     interfaces
br-eth1         8000.000d88525125       no              eth1
                                                        qtap1
br-eth3         8000.000d88525127       no              eth3
                                                        qtap3

This is the way I start the guest:
kvm -net nic,vlan=1,model=rtl8139,macaddr=52:54:00:12:34:56 -net tap,vlan=1,ifname=qtap1,script=no,downscript=no -net nic,vlan=3,model=rtl8139,macaddr=52:54:00:12:34:58 -net tap,vlan=3,ifname=qtap3,script=no,downscript=no Robbie.img -m 1024

The eth3/br-eth3/qtap3 looks OK (I can ping the "default-GW" on that network from the guest) but the connection to the "shared" interface (eth1/br- eth1/qtap1) does not work, I can not ping or ssh to/from the guest from/to the host. Do not ask me if I can ping any other host on that network - there are no other host on the network yet, just the Host and the guest.

Any suggestions?

Am 04.04.2010 20:02, schrieb Dan Johansson:

Quoted 146 lines · On Sunday 04 April 2010 15.00:26 sudhir kumar wrote:

Quoted 54 lines · Dan Johansson

Quoted 48 lines

Hi,

I am new to this list and to KVM (and qemu) so please be gentle with me. Up until now I have been running my virtualizing using VMWare-Server. Now I want to try KVM due to some issues with the VMWare-Server and I am having some troubles with the networking part of KVM.

This is a small example of what I want (best viewed in a fix-font):

+-----------------------------------+

 | Host                              |
 |  +----------+                eth0 |---- 192.168.1.0/24
 |
 |  |      eth0|-- +                 |
 |  | VM1  eth1|---(---+------- eth1 |---- 192.168.2.0/24
 |  |      eth2|---(---(---+         |
 |
 |  +----------+   |   |   |         |
 |
 |  +----------+   +---(---(--- eth2 |---- 192.168.1.0/24
 |
 |  |      eth0|---+   |   |         |
 |  | VM2  eth1|-------+   +--- eth3 |---- 192.168.3.0/24
 |  |      eth2|-----------+         |
 |
 |  +----------+                     |

+-----------------------------------+

Host-eth0 is only for the Host (no VM)
Host-eth1 is shared between the Host and the VM's (VM?-eth1) Host-eth2 and Host-eth3 are only for the VMs (eth0 and eth2)

The Host and the VMs all have fixed IPs (no dhcp or likewise). In this example th IPs could be:

Host-eth0:      192.168.1.1
Host-eth1:      192.168.2.1
Host-eth2:      -
Host-eth3:      -
VM1-eth0:               192.168.1.11
VM1-eth1:               192.168.2.11
VM1-eth2:               192.168.3.11
VM2-eth0:               192.168.1.22
VM2-eth1:               192.168.2.22
VM3-eth2:               192.168.3.22

Quoted 4 lines

And, yes, Host-eth0 and Host-eth2 are in the same subnet, with eth0 dedicated to the Host and eth2 dedicated to the VMs.

In VMWare this was quite easy to setup (three bridged networks).

Quoted 12 lines

Its easy with KVM too. You want 3 NICs per VM, so you need to pass the corresponding parameters(including qemu-ifup script) for 3 NICs to each VM.
In the host you need to create 2 bridges: say br-eth1 and br-eth2. Make them as the interface on the host in place of the corresponding eth interfaces.(brct addbr br-eth1; ifcfg eth1 0.0.0.0 up; brctl addif br-eth eth1; assign eth1's ip and routes to breth1; same for eth2). In the corresponding qemu-ifup scripts of each interface use bridge=br-ethN (This basicaly translates to brctl addif br-ethN $1, where $ is the tap device created)
This should work perfectly fine with your existing NW setup. For a quick reference use: http://www.linux-kvm.org/page/Networking

Thanks for your help, but... I am still not able to get it to work the way I want.
This is what I have don so far:
brctl addbr br-eth1
brctl addbr br-eth3

ip link set eth1 up
ip link set eth3 up

brctl addif br-eth1 eth1
brctl addif br-eth3 eth3

tunctl -b -t qtap1
tunctl -b -t qtap3

brctl addif br-eth1 qtap1
brctl addif br-eth3 qtap3

ifconfig qtap1 up 0.0.0.0 promisc
ifconfig qtap3 up 0.0.0.0 promisc

# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0d:88:52:51:24
          inet addr:192.168.1.3  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:443638 errors:0 dropped:0 overruns:0 frame:0
          TX packets:758540 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:47041686 (44.8 MiB)  TX bytes:990115354 (944.2 MiB)
          Interrupt:19 Base address:0xec00

eth1 Link encap:Ethernet HWaddr 00:0d:88:52:51:25
inet addr:192.168.4.1 Bcast:192.168.4.255 Mask:255.255.255.0 UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:6 collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:360 (360.0 B) Interrupt:18 Base address:0xe880

eth3 Link encap:Ethernet HWaddr 00:0d:88:52:51:27
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:240 (240.0 B) Interrupt:16 Base address:0xe480

qtap1 Link encap:Ethernet HWaddr 26:c0:de:df:c5:e4
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:351 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500
RX bytes:14742 (14.3 KiB) TX bytes:0 (0.0 B)

qtap3 Link encap:Ethernet HWaddr 26:3e:ba:2d:97:bc
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500
RX bytes:252 (252.0 B) TX bytes:0 (0.0 B)

# brctl show
bridge name     bridge id               STP enabled     interfaces
br-eth1         8000.000d88525125       no              eth1
                                                        qtap1
br-eth3         8000.000d88525127       no              eth3
                                                        qtap3

This is the way I start the guest:
kvm -net nic,vlan=1,model=rtl8139,macaddr=52:54:00:12:34:56 -net tap,vlan=1,ifname=qtap1,script=no,downscript=no -net nic,vlan=3,model=rtl8139,macaddr=52:54:00:12:34:58 -net tap,vlan=3,ifname=qtap3,script=no,downscript=no Robbie.img -m 1024

Quoted 8 lines

The eth3/br-eth3/qtap3 looks OK (I can ping the "default-GW" on that network from the guest) but the connection to the "shared" interface (eth1/br- eth1/qtap1) does not work, I can not ping or ssh to/from the guest from/to the host. Do not ask me if I can ping any other host on that network - there are no other host on the network yet, just the Host and the guest.

Any suggestions?

eth1 should not have an IP address:

# ifconfig eth1 0.0.0.0

br-eth1 is not activated (it's missing in ifconfig), and it needs an IP address:

# ifconfig br-eth1 192.168.2.1/24

Even if it works I would explicitly activate br-eth3 too:

# ifconfig br-eth3 0.0.0.0 up

Looking at the output of ifconfig shows that the IP-address of eth0 (192.168.1.3) doesn't match 192.168.1.1 from your address list, and eth1 (192.168.4.1) is in a different network than the specified 192.168.2.1/24.

Hi,

For networks 1 and 2 (respectively 192.168.1.0/24 and 192.168.2.0/24, I'll acll also network 3 for the last one) you'll have to use bridges. For network 3 you can use vde which is quite simply to use and gives possibility to add new virtual networks without the need of physical interface.

For bridges configuration:
brctl addbr ${brname}
brctl setfd ${brname} 0
brctl sethello ${brname} 1
brctl stp ${brname} off
brct addif ${brname} ${physical_if}

Here you can add an IP to you're bridge interface if needed for the host
has access to this network. Host's IP must by set up on bridge interface with configuration, if IP is set up on physical interface it is not working. I'm wondering why, my network knowledge is not much :/

Then when you'll run qemu to set up a new VM you'll have to use a tap
device for the VM

One important thing when use several networks in the same host is then vlan ID you'll use for networks. These vlans aren't (from my understanding : ) 802.1Q vlans but pure qemu/kvm concepts. If you don't set up each VM's interface on the right kvm's vlan they will all be set up on vlan number 0, which means packets from all networks will appear everywhere (tcpdump on 192.168.1.0/24 card will see traffic for others networks).

Here is the command I use to launch a VM with two bridged interfaces and
one other on some vde network:
screen -d -m qemu-system-x86_64 -m 64 \

(quoted diff) +0 -8

-net tap,vlan=0,ifname=tap0,script=/path/to/scripts/kvm-ifup.br0 \
-net nic,vlan=0,model=e1000,macaddr=52:54:00:01:ab:cd \
-net tap,vlan=3,ifname=tap1,script=/path/to/scripts/kvm-ifup.br1 \
-net nic,vlan=3,model=e1000,macaddr=52:54:00:03:ab:cd \
-net vde,vlan=2,sock=/var/vde/switches/vdelan0 \
-net nic,vlan=2,model=e1000,macaddr=52:54:00:02:ab:cd \
-hda /path/to/disks/vm.vmdk \
-nographic

Note the "vlan=X" option set on both -net arguments for each interface.

For VDE switchs:
/usr/bin/vde_switch -m 770 -g qemu -daemon -n 8 -t dummy10 -s /var/vde/switches/vdelan0 -M /var/vde/management/vdelan0

I use dummy interface on host to set up vde networks because the host mustn't have access on these networks.

(quoted diff) +0 -3

-s directory where VDE's unix sockets will be store for this network
-M management socket (I haven't yet understood how to use them :p)
-n number of port on the switch

other options are quite easy to understand.

Finally my scripts to set up bridged interface are the following:
------------------
#!/bin/sh
#set -x
switch=echo $0 | awk -F\. '{ print $NF}'
if [ -n "$1" ];then
#/usr/bin/tunctl -u whoami -t $1
/sbin/ip link set $1 up
sleep 0.5s
/sbin/brctl addif $switch $1
exit 0
else
echo "Error: no interface specified"
exit 1
fi
------------------
I use one script per bridge named kvm-ifup.${brname} because the script's name define the bridge on which the interface will be attached
(kvm-ifup.br0 -> br0 and kvm-ifup.br1 -> br1).

I use VDE switch for networks on which host isn't plugged because they are simpler to use for me but it is not the only way to proceed.

A last thing, I have a physical card on which is plugged my modem. As I don't want to expose the host on Internet I use a VM to act as a firewall. As I just can't give this VM direct access on the physical card dedicated
to Internet I use a bridge:

modem ------- physical card ----- bridge ----- tap ---- VM's internal card

The modem knows only one MAC address: the one from physical interface but this physical interface as no IP. The bridge has same MAC address as physical interface but has no more IP. The card with an IP is the VM's internal card so for packets pass through all these interface without IP I use ebtables to "nat" MAC address between physical interface and VM's internal card:
ebtables -t nat -A PREROUTING -i $PHYS_DEV_NAME -d $PHYS_DEV_MAC -j dnat --to-destination $VIRT_DEV_MAC
ebtables -t nat -A POSTROUTING -o $VIRT_DEV_NAME -s $VIRT_DEV_MAC -j snat --to-source $PHYS_DEV_MAC

Finally there are some interesting links:
forum's thread related to vlan issues
http://serverfault.com/questions/101477/issue-with-multiple-bridging-for-kv
m-hosts

something about bridges:
http://www.savelono.com/linux/using-multiple-interfaces-with-kvm-and-xen.ht
ml

different method to build networks using qemu/kvm:
http://people.gnome.org/~markmc/qemu-networking.html<http://people.gnome.or
g/%7Emarkmc/qemu-networking.html>

Hoping this helps.

Kindly regards,

mathias

2010/4/5 Dan Johansson <[email protected]>

Quoted 225 lines · On Sunday 04 April 2010 22.23:28 Held Bernhard wrote:

Quoted 10 lines

Am 04.04.2010 20:02, schrieb Dan Johansson:

Quoted 9 lines · On Sunday 04 April 2010 15.00:26 sudhir kumar wrote:

Quoted 8 lines · Dan Johansson

Quoted 7 lines

Hi,

I am new to this list and to KVM (and qemu) so please be gentle with me. Up until now I have been running my virtualizing using VMWare-Server. Now I want to try KVM due to some issues with the VMWare-Server and I am having some troubles with the networking part

of

Quoted 78 lines

Quoted 78 lines

Quoted 47 lines

Quoted 42 lines

KVM.

This is a small example of what I want (best viewed in a fix-font):

+-----------------------------------+

 | Host                              |
 |  +----------+                eth0 |---- 192.168.1.0/24
 |
 |  |      eth0|-- +                 |
 |  | VM1  eth1|---(---+------- eth1 |---- 192.168.2.0/24
 |  |      eth2|---(---(---+         |
 |
 |  +----------+   |   |   |         |
 |
 |  +----------+   +---(---(--- eth2 |---- 192.168.1.0/24
 |
 |  |      eth0|---+   |   |         |
 |  | VM2  eth1|-------+   +--- eth3 |---- 192.168.3.0/24
 |  |      eth2|-----------+         |
 |
 |  +----------+                     |

+-----------------------------------+

Host-eth0 is only for the Host (no VM)
Host-eth1 is shared between the Host and the VM's (VM?-eth1) Host-eth2 and Host-eth3 are only for the VMs (eth0 and eth2)

The Host and the VMs all have fixed IPs (no dhcp or likewise). In this example th IPs could be:

Host-eth0:      192.168.1.1
Host-eth1:      192.168.2.1
Host-eth2:      -
Host-eth3:      -
VM1-eth0:               192.168.1.11
VM1-eth1:               192.168.2.11
VM1-eth2:               192.168.3.11
VM2-eth0:               192.168.1.22
VM2-eth1:               192.168.2.22
VM3-eth2:               192.168.3.22

Quoted 4 lines

And, yes, Host-eth0 and Host-eth2 are in the same subnet, with eth0 dedicated to the Host and eth2 dedicated to the VMs.

In VMWare this was quite easy to setup (three bridged networks).

Quoted 12 lines

Its easy with KVM too. You want 3 NICs per VM, so you need to pass the corresponding parameters(including qemu-ifup script) for 3 NICs to each VM.
In the host you need to create 2 bridges: say br-eth1 and br-eth2. Make them as the interface on the host in place of the corresponding eth interfaces.(brct addbr br-eth1; ifcfg eth1 0.0.0.0 up; brctl addif br-eth eth1; assign eth1's ip and routes to breth1; same for eth2). In the corresponding qemu-ifup scripts of each interface use bridge=br-ethN (This basicaly translates to brctl addif br-ethN $1, where $ is the tap device created)
This should work perfectly fine with your existing NW setup. For a quick reference use: http://www.linux-kvm.org/page/Networking

Thanks for your help, but... I am still not able to get it to work the way I want.
This is what I have don so far:
brctl addbr br-eth1
brctl addbr br-eth3

ip link set eth1 up
ip link set eth3 up

brctl addif br-eth1 eth1
brctl addif br-eth3 eth3

tunctl -b -t qtap1
tunctl -b -t qtap3

brctl addif br-eth1 qtap1
brctl addif br-eth3 qtap3

Quoted 6 lines

Quoted 6 lines

ifconfig qtap1 up 0.0.0.0 promisc
ifconfig qtap3 up 0.0.0.0 promisc

# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0d:88:52:51:24
          inet addr:192.168.1.3  Bcast:192.168.1.255

Mask:255.255.255.0

Quoted 9 lines

Quoted 9 lines

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:443638 errors:0 dropped:0 overruns:0 frame:0
          TX packets:758540 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:47041686 (44.8 MiB)  TX bytes:990115354 (944.2

MiB) Interrupt:19 Base address:0xec00

eth1 Link encap:Ethernet HWaddr 00:0d:88:52:51:25

          inet addr:192.168.4.1  Bcast:192.168.4.255

Mask:255.255.255.0

Quoted 45 lines

Quoted 45 lines

UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:6 collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:360 (360.0 B) Interrupt:18 Base address:0xe880

eth3 Link encap:Ethernet HWaddr 00:0d:88:52:51:27
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:240 (240.0 B) Interrupt:16 Base address:0xe480

qtap1 Link encap:Ethernet HWaddr 26:c0:de:df:c5:e4
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:351 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500
RX bytes:14742 (14.3 KiB) TX bytes:0 (0.0 B)

qtap3 Link encap:Ethernet HWaddr 26:3e:ba:2d:97:bc
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500
RX bytes:252 (252.0 B) TX bytes:0 (0.0 B)

# brctl show
bridge name     bridge id               STP enabled     interfaces
br-eth1         8000.000d88525125       no              eth1
                                                        qtap1
br-eth3         8000.000d88525127       no              eth3
                                                        qtap3

This is the way I start the guest:
kvm -net nic,vlan=1,model=rtl8139,macaddr=52:54:00:12:34:56 -net tap,vlan=1,ifname=qtap1,script=no,downscript=no -net nic,vlan=3,model=rtl8139,macaddr=52:54:00:12:34:58 -net tap,vlan=3,ifname=qtap3,script=no,downscript=no Robbie.img -m 1024

Quoted 6 lines

Quoted 6 lines

The eth3/br-eth3/qtap3 looks OK (I can ping the "default-GW" on that network from the guest) but the connection to the "shared" interface (eth1/br- eth1/qtap1) does not work, I can not ping or ssh to/from the guest from/to the host. Do not ask me if I can ping any other host on that network - there are no other host on the network yet, just the

Host

Quoted 9 lines

Quoted 3 lines

and the guest.

Any suggestions?

eth1 should not have an IP address:

# ifconfig eth1 0.0.0.0

br-eth1 is not activated (it's missing in ifconfig), and it needs an IP address:

Quoted 8 lines

# ifconfig br-eth1 192.168.2.1/24

Even if it works I would explicitly activate br-eth3 too:

# ifconfig br-eth3 0.0.0.0 up

Looking at the output of ifconfig shows that the IP-address of eth0 (192.168.1.3) doesn't match 192.168.1.1 from your address list, and eth1 (192.168.4.1) is in a different network than the specified

192.168.2.1/24.

OK, the 192.168.1.3 - 192.168.1.1 is my typo, 192.168.1.3 is the correct IP for the Host on this interface.
I was first planning to use the 192.168.2.0/24 network here, but I had forgotten that that was already used somewhere else in my network, so I just
picked the next "free" network address. Sorry for the confusion.

I have tried you suggestions above but I still have no network connection.

# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:0d:88:52:51:25
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3 errors:0 dropped:0 overruns:0 carrier:3
         collisions:0 txqueuelen:1000
         RX bytes:0 (0.0 B)  TX bytes:180 (180.0 B)
          Interrupt:18 Base address:0xe880

# ifconfig br-eth1
br-eth1   Link encap:Ethernet  HWaddr 00:0d:88:52:51:25
          inet addr:192.168.4.1  Bcast:192.168.4.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:65 errors:0 dropped:0 overruns:0 frame:0
         TX packets:62 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:3518 (3.4 KiB)  TX bytes:4750 (4.6 KiB)

# ifconfig br-eth3
br-eth3   Link encap:Ethernet  HWaddr 00:0d:88:52:51:27
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
         RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Regards,

"external", VM <--> "external")

From the Host to the VM:

Quoted 111 lines · On Monday 05 April 2010 13.34:26 mathias dufresne wrote:

Hi,

For networks 1 and 2 (respectively 192.168.1.0/24 and 192.168.2.0/24, I'll acll also network 3 for the last one) you'll have to use bridges. For network 3 you can use vde which is quite simply to use and gives possibility to add new virtual networks without the need of physical interface.

For bridges configuration:
brctl addbr ${brname}
brctl setfd ${brname} 0
brctl sethello ${brname} 1
brctl stp ${brname} off
brct addif ${brname} ${physical_if}

Here you can add an IP to you're bridge interface if needed for the host
has access to this network. Host's IP must by set up on bridge interface with configuration, if IP is set up on physical interface it is not working. I'm wondering why, my network knowledge is not much :/

Then when you'll run qemu to set up a new VM you'll have to use a tap
device for the VM

One important thing when use several networks in the same host is then vlan ID you'll use for networks. These vlans aren't (from my understanding : ) 802.1Q vlans but pure qemu/kvm concepts. If you don't set up each VM's interface on the right kvm's vlan they will all be set up on vlan number 0, which means packets from all networks will appear everywhere (tcpdump on 192.168.1.0/24 card will see traffic for others networks).

Here is the command I use to launch a VM with two bridged interfaces and
one other on some vde network:
screen -d -m qemu-system-x86_64 -m 64 \

(quoted diff) +0 -8

-net tap,vlan=0,ifname=tap0,script=/path/to/scripts/kvm-ifup.br0 \
-net nic,vlan=0,model=e1000,macaddr=52:54:00:01:ab:cd \
-net tap,vlan=3,ifname=tap1,script=/path/to/scripts/kvm-ifup.br1 \
-net nic,vlan=3,model=e1000,macaddr=52:54:00:03:ab:cd \
-net vde,vlan=2,sock=/var/vde/switches/vdelan0 \
-net nic,vlan=2,model=e1000,macaddr=52:54:00:02:ab:cd \
-hda /path/to/disks/vm.vmdk \
-nographic

Note the "vlan=X" option set on both -net arguments for each interface.

For VDE switchs:
/usr/bin/vde_switch -m 770 -g qemu -daemon -n 8 -t dummy10 -s /var/vde/switches/vdelan0 -M /var/vde/management/vdelan0

I use dummy interface on host to set up vde networks because the host mustn't have access on these networks.

(quoted diff) +0 -3

-s directory where VDE's unix sockets will be store for this network
-M management socket (I haven't yet understood how to use them :p)
-n number of port on the switch

other options are quite easy to understand.

Finally my scripts to set up bridged interface are the following:
------------------
#!/bin/sh
#set -x
switch=echo $0 | awk -F\. '{ print $NF}'
if [ -n "$1" ];then
#/usr/bin/tunctl -u whoami -t $1
/sbin/ip link set $1 up
sleep 0.5s
/sbin/brctl addif $switch $1
exit 0
else
echo "Error: no interface specified"
exit 1
fi
------------------
I use one script per bridge named kvm-ifup.${brname} because the script's name define the bridge on which the interface will be attached
(kvm-ifup.br0 -> br0 and kvm-ifup.br1 -> br1).

I use VDE switch for networks on which host isn't plugged because they are simpler to use for me but it is not the only way to proceed.

A last thing, I have a physical card on which is plugged my modem. As I don't want to expose the host on Internet I use a VM to act as a firewall. As I just can't give this VM direct access on the physical card dedicated
to Internet I use a bridge:

modem ------- physical card ----- bridge ----- tap ---- VM's internal card

The modem knows only one MAC address: the one from physical interface but this physical interface as no IP. The bridge has same MAC address as physical interface but has no more IP. The card with an IP is the VM's internal card so for packets pass through all these interface without IP I use ebtables to "nat" MAC address between physical interface and VM's internal card:
ebtables -t nat -A PREROUTING -i $PHYS_DEV_NAME -d $PHYS_DEV_MAC -j dnat --to-destination $VIRT_DEV_MAC
ebtables -t nat -A POSTROUTING -o $VIRT_DEV_NAME -s $VIRT_DEV_MAC -j snat --to-source $PHYS_DEV_MAC

Finally there are some interesting links:
forum's thread related to vlan issues
http://serverfault.com/questions/101477/issue-with-multiple-bridging-for-kv
m-hosts

something about bridges:
http://www.savelono.com/linux/using-multiple-interfaces-with-kvm-and-xen.ht
ml

different method to build networks using qemu/kvm:
http://people.gnome.org/~markmc/qemu-networking.html<http://people.gnome.or
g/%7Emarkmc/qemu-networking.html>

Quoted 233 lines

Hoping this helps.

Kindly regards,

mathias

2010/4/5 Dan Johansson<[email protected]>

Quoted 225 lines · On Sunday 04 April 2010 22.23:28 Held Bernhard wrote:

Quoted 10 lines

Am 04.04.2010 20:02, schrieb Dan Johansson:

Quoted 9 lines · On Sunday 04 April 2010 15.00:26 sudhir kumar wrote:

Quoted 8 lines · Dan Johansson

Quoted 7 lines

Hi,

I am new to this list and to KVM (and qemu) so please be gentle with me. Up until now I have been running my virtualizing using VMWare-Server. Now I want to try KVM due to some issues with the VMWare-Server and I am having some troubles with the networking part

of

Quoted 78 lines

Quoted 78 lines

Quoted 47 lines

Quoted 42 lines

KVM.

This is a small example of what I want (best viewed in a fix-font):

+-----------------------------------+

  | Host                              |
  |  +----------+                eth0 |---- 192.168.1.0/24
  |
  |  |      eth0|-- +                 |
  |  | VM1  eth1|---(---+------- eth1 |---- 192.168.2.0/24
  |  |      eth2|---(---(---+         |
  |
  |  +----------+   |   |   |         |
  |
  |  +----------+   +---(---(--- eth2 |---- 192.168.1.0/24
  |
  |  |      eth0|---+   |   |         |
  |  | VM2  eth1|-------+   +--- eth3 |---- 192.168.3.0/24
  |  |      eth2|-----------+         |
  |
  |  +----------+                     |

+-----------------------------------+

Host-eth0 is only for the Host (no VM)
Host-eth1 is shared between the Host and the VM's (VM?-eth1) Host-eth2 and Host-eth3 are only for the VMs (eth0 and eth2)

The Host and the VMs all have fixed IPs (no dhcp or likewise). In this example th IPs could be:

Host-eth0:      192.168.1.1
Host-eth1:      192.168.2.1
Host-eth2:      -
Host-eth3:      -
VM1-eth0:               192.168.1.11
VM1-eth1:               192.168.2.11
VM1-eth2:               192.168.3.11
VM2-eth0:               192.168.1.22
VM2-eth1:               192.168.2.22
VM3-eth2:               192.168.3.22

Quoted 4 lines

And, yes, Host-eth0 and Host-eth2 are in the same subnet, with eth0 dedicated to the Host and eth2 dedicated to the VMs.

In VMWare this was quite easy to setup (three bridged networks).

Quoted 12 lines

Its easy with KVM too. You want 3 NICs per VM, so you need to pass the corresponding parameters(including qemu-ifup script) for 3 NICs to each VM.
In the host you need to create 2 bridges: say br-eth1 and br-eth2. Make them as the interface on the host in place of the corresponding eth interfaces.(brct addbr br-eth1; ifcfg eth1 0.0.0.0 up; brctl addif br-eth eth1; assign eth1's ip and routes to breth1; same for eth2). In the corresponding qemu-ifup scripts of each interface use bridge=br-ethN (This basicaly translates to brctl addif br-ethN $1, where $ is the tap device created)
This should work perfectly fine with your existing NW setup. For a quick reference use: http://www.linux-kvm.org/page/Networking

Thanks for your help, but... I am still not able to get it to work the way I want.
This is what I have don so far:
brctl addbr br-eth1
brctl addbr br-eth3

ip link set eth1 up
ip link set eth3 up

brctl addif br-eth1 eth1
brctl addif br-eth3 eth3

tunctl -b -t qtap1
tunctl -b -t qtap3

brctl addif br-eth1 qtap1
brctl addif br-eth3 qtap3

Quoted 6 lines

Quoted 6 lines

ifconfig qtap1 up 0.0.0.0 promisc
ifconfig qtap3 up 0.0.0.0 promisc

# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0d:88:52:51:24
           inet addr:192.168.1.3  Bcast:192.168.1.255

Mask:255.255.255.0

Quoted 9 lines

Quoted 9 lines

           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:443638 errors:0 dropped:0 overruns:0 frame:0
           TX packets:758540 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:47041686 (44.8 MiB)  TX bytes:990115354 (944.2

MiB) Interrupt:19 Base address:0xec00

eth1 Link encap:Ethernet HWaddr 00:0d:88:52:51:25

           inet addr:192.168.4.1  Bcast:192.168.4.255

Mask:255.255.255.0

Quoted 45 lines

Quoted 45 lines

UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:6 collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:360 (360.0 B) Interrupt:18 Base address:0xe880

eth3 Link encap:Ethernet HWaddr 00:0d:88:52:51:27
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:240 (240.0 B) Interrupt:16 Base address:0xe480

qtap1 Link encap:Ethernet HWaddr 26:c0:de:df:c5:e4
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:351 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500
RX bytes:14742 (14.3 KiB) TX bytes:0 (0.0 B)

qtap3 Link encap:Ethernet HWaddr 26:3e:ba:2d:97:bc
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500
RX bytes:252 (252.0 B) TX bytes:0 (0.0 B)

# brctl show
bridge name     bridge id               STP enabled     interfaces
br-eth1         8000.000d88525125       no              eth1
                                                         qtap1
br-eth3         8000.000d88525127       no              eth3
                                                         qtap3

This is the way I start the guest:
kvm -net nic,vlan=1,model=rtl8139,macaddr=52:54:00:12:34:56 -net tap,vlan=1,ifname=qtap1,script=no,downscript=no -net nic,vlan=3,model=rtl8139,macaddr=52:54:00:12:34:58 -net tap,vlan=3,ifname=qtap3,script=no,downscript=no Robbie.img -m 1024

Quoted 6 lines

Quoted 6 lines

The eth3/br-eth3/qtap3 looks OK (I can ping the "default-GW" on that network from the guest) but the connection to the "shared" interface (eth1/br- eth1/qtap1) does not work, I can not ping or ssh to/from the guest from/to the host. Do not ask me if I can ping any other host on that network - there are no other host on the network yet, just the

Host

Quoted 9 lines

Quoted 3 lines

and the guest.

Any suggestions?

eth1 should not have an IP address:

# ifconfig eth1 0.0.0.0

br-eth1 is not activated (it's missing in ifconfig), and it needs an IP address:

Quoted 8 lines

# ifconfig br-eth1 192.168.2.1/24

Even if it works I would explicitly activate br-eth3 too:

# ifconfig br-eth3 0.0.0.0 up

Looking at the output of ifconfig shows that the IP-address of eth0 (192.168.1.3) doesn't match 192.168.1.1 from your address list, and eth1 (192.168.4.1) is in a different network than the specified

192.168.2.1/24.

OK, the 192.168.1.3 - 192.168.1.1 is my typo, 192.168.1.3 is the correct IP for the Host on this interface.
I was first planning to use the 192.168.2.0/24 network here, but I had forgotten that that was already used somewhere else in my network, so I just
picked the next "free" network address. Sorry for the confusion.

I have tried you suggestions above but I still have no network connection.

# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:0d:88:52:51:25
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:3 errors:0 dropped:0 overruns:0 carrier:3
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:180 (180.0 B)
           Interrupt:18 Base address:0xe880

# ifconfig br-eth1
br-eth1   Link encap:Ethernet  HWaddr 00:0d:88:52:51:25
           inet addr:192.168.4.1  Bcast:192.168.4.255  Mask:255.255.255.0
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:65 errors:0 dropped:0 overruns:0 frame:0
          TX packets:62 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3518 (3.4 KiB)  TX bytes:4750 (4.6 KiB)

# ifconfig br-eth3
br-eth3   Link encap:Ethernet  HWaddr 00:0d:88:52:51:27
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Regards,

OK, lets simplify it a bit. This is now the layout that I am playing with and trying to get KVM to work .

  +-----------------------------------+
   | Host                              |
   |  +----------+                eth0 |---- 192.168.1.0/24
   |  | VM   eth1|------qtap1          |
   |  |          |        |            |
   |  | eth0     |      br-eth1---eth1 |---- 192.168.4.0/24
   |  +----------+                     |
   |                                   |
   +-----------------------------------+

Host-eth0 is only for the Host (no VM I/O)
Host-eth1 is shared between the Host and the VM (Host<--> VM, Host<--

Quoted 1 line

"external", VM<--> "external")

VM-eth0 is unused at the moment.

This is the commands I have been using to setup the Networking:
brctl addbr br-eth1
brctl setfd br-eth1 0
brctl sethello br-eth1 1
brctl stp br-eth1 off
brctl addif br-eth1 eth1
tunctl -b -t qtap1
brctl addif br-eth1 qtap1
ifconfig qtap1 up 0.0.0.0 promisc
ifconfig br-eth1 192.168.4.1/24 up
ifconfig eth1 0.0.0.0 up

And this is the way I start the VM:
kvm -net nic,vlan=1,model=rtl8139,macaddr=52:54:00:12:34:56 -net tap,vlan=1,ifname=qtap1,script=no,downscript=no Robbie.img -m 1024 -k de-ch

On the Host I have the following:

# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:0d:88:52:51:25
           UP BROADCAST MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:18 errors:0 dropped:0 overruns:0 carrier:18
           collisions:0 txqueuelen:1000
           RX bytes:0 (0.0 B)  TX bytes:1080 (1.0 KiB)
           Interrupt:18 Base address:0xe880

# ifconfig br-eth1
br-eth1   Link encap:Ethernet  HWaddr 00:0d:88:52:51:25
           inet addr:192.168.4.1  Bcast:192.168.4.255  Mask:255.255.255.0
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:501 errors:0 dropped:0 overruns:0 frame:0
           TX packets:479 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:26378 (25.7 KiB)  TX bytes:35080 (34.2 KiB)

# ifconfig qtap1
qtap1     Link encap:Ethernet  HWaddr da:77:3a:a9:40:23
           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
           RX packets:503 errors:0 dropped:0 overruns:0 frame:0
           TX packets:481 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:500
           RX bytes:33524 (32.7 KiB)  TX bytes:35240 (34.4 KiB)

# brctl show
bridge name     bridge id               STP enabled     interfaces
br-eth1         8000.000d88525125       no              eth1
                                                         qtap1
# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
192.168.4.0     0.0.0.0         255.255.255.0   U         0 0          0 br-
eth1
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0
239.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 eth0
127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo
0.0.0.0         192.168.1.1     0.0.0.0         UG        0 0          0 eth0

And in the VM I have:

# ifconfig
eth1      Link encap:Ethernet  HWaddr 52:54:00:12:34:56
           inet addr:192.168.4.4  Bcast:192.168.4.255  Mask:255.255.255.0
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:12 errors:0 dropped:0 overruns:0 frame:0
           TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:1040 (1.0 KiB)  TX bytes:1018 (1018.0 B)
           Interrupt:11 Base address:0xc000

# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
192.168.4.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1
127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo
0.0.0.0         192.168.4.1     0.0.0.0         UG        0 0          0 eth1

But Still I get the following:

Quoted 1 line

From the Host to the VM:

$ ssh [email protected]
ssh: connect to host 192.168.4.4 port 22: Connection refused

And from the VM to the Host:

# ssh [email protected]
ssh: connect to host 192.168.4.1 port 22: Connection refused

Any more suggestions?

Hi Dan!

Quoted 10 lines

But Still I get the following:
From the Host to the VM:

$ ssh [email protected]
ssh: connect to host 192.168.4.4 port 22: Connection refused

And from the VM to the Host:

# ssh [email protected]
ssh: connect to host 192.168.4.1 port 22: Connection refused

Any more suggestions?

qtap1 and eth1 on the guest show rx- and tx-packets, this looks not that bad.

"Connection refused" smells like a closed or firewalled port (hosts.allow / hosts.deny?). If there's no network at all (and no arp resolution) ssh would print "no route to host".

Please make sure that you remove all rules from iptables on both the host and the guest. Then start pinging:
host -> host
host -> guest
guest -> guest
guest -> host

If I have to debug network problems I always run tcpdump. This way I can quickly isolate the problem (tcpdump -i eth1, tcpdump -i qtap1).

Am 05.04.2010 17:31, schrieb Held Bernhard:

Quoted 30 lines

Hi Dan!

Quoted 10 lines

But Still I get the following:
From the Host to the VM:

$ ssh [email protected]
ssh: connect to host 192.168.4.4 port 22: Connection refused

And from the VM to the Host:

# ssh [email protected]
ssh: connect to host 192.168.4.1 port 22: Connection refused

Any more suggestions?

qtap1 and eth1 on the guest show rx- and tx-packets, this looks not that bad.

"Connection refused" smells like a closed or firewalled port (hosts.allow / hosts.deny?). If there's no network at all (and no arp resolution) ssh would print "no route to host".

Please make sure that you remove all rules from iptables on both the host and the guest. Then start pinging:
host -> host
host -> guest
guest -> guest
guest -> host

If I have to debug network problems I always run tcpdump. This way I can quickly isolate the problem (tcpdump -i eth1, tcpdump -i qtap1).

Quoted 2 lines

Regards,
Bernhard

Just another idea: did you enable routing?

echo 1 > /proc/sys/net/ipv4/ip_forward

Must I specify an IP for the br-eth3 interface?

Regards,

This should be done over the host-eth3 interface and I have set up the br-eth3 and qtap3 the same way as with the eth1/br-eth1/qtap1 with one difference - the br-eth3 interface is setup without an IP.
When doing traffic from the VM to the network I can see traffic on the qtap3 and br-eth3 interface but none on the eth3.

Must I specify an IP for the br-eth3 interface?

Hi Dan!

Quoted 6 lines

This should be done over the host-eth3 interface and I have set up the br-eth3 and qtap3 the same way as with the eth1/br-eth1/qtap1 with one difference - the br-eth3 interface is setup without an IP. When doing traffic from the VM to the network I can see traffic on the qtap3 and br-eth3 interface but none on the eth3.

Quoted 1 line

Must I specify an IP for the br-eth3 interface?

No, an IP is not required. Just activate br-eth3 and don't forget to add a route:

ifconfig br-eth3 up 0.0.0.0
route add 192.168.3.0 netmask 255.255.255.0 eth3