KVM VM's facing public network

DISCUSSION Hugo R Hernández-Mora 2 participants 4 messages Jan 29, 2013 KVM

Share & tools

Hugo R Hernández-Mora

author

2013-01-29 17:54 UTC #0 lore raw

Hello there,
we are experiencing a problem by configuring a KVM bridged networking to share a public network interface between the KVM host and the VMs. Currently, our KVM server has set three network interfaces as follows:

* eth0: 192.168.10.101/23 (main interface for public network - no bridge)
* eth1 <--> br1: 192.168.10.201/23 (KVM VMs connected to public network)
* eth3 <--> br3: 10.7.10.201/23 (KVM VMs connected to LAN)

We have followed instructions as from Red Hat as well as from diferrent web sites and we are not able to get the VMs to get access into/from the public network. Here is a more detailed configuration for the KVM host:

ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
HWADDR=AC:80:B2:14:C5:EE
BOOTPROTO=none
IPADDR=192.168.10.101
NETMASK=255.255.254.0

ifcfg-eth1
DEVICE=eth1
ONBOOT=yes
HWADDR=AC:80:B2:4E:D3:28
BRIDGE=br1

ifcfg-br1
DEVICE=br1
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=none
IPADDR=192.168.10.201
NETMASK=255.255.254.0
STP=off
DELAY=0

ifcfg-eth3
DEVICE=eth3
ONBOOT=yes
HWADDR=AC:80:B2:4E:D3:2A
BRIDGE=br3

ifcfg-br3
DEVICE=br3
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=static
IPADDR=10.7.10.201
NETMASK=255.255.254.0
STP=off
DELAY=0

network
NETWORKING=yes
HOSTNAME=kvm1.public-lan.net
GATEWAY=192.168.10.1

For iptables/routing, we have followed instructions as explained on http://www.linux-kvm.org/page/Networking#public_bridge
*nat
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING --out-interface br1 -j MASQUERADE
COMMIT
:FORWARD ACCEPT [0:0]
-A FORWARD --in-interface br1 -j ACCEPT

Hostside:
Allow IPv4 forwarding and add route to client (could be put in a script - route has to be added after the client has started):
sysctl -w net.ipv4.ip_forward=1 # allow forwarding of IPv4
route add -host <ip-of-client> dev <tap-device> # add route to the client

Clientside:
Default GW of the client is of course then the host (<ip-of-host> has to be in same subnet as <ip-of-client> ...):
route add default gw <ip-of-host>

But it doesn't seem to work. My assumption the problem is related with a wrong setting of the firewall on the iptables. Could you please advice? Your help will be greatly appreciated!

We are running Scientific Linux 6.2 on the KVM server as well as on the VMs. There is no network issue by accessing the LAN between VMs but only to face the public network.

Thanks in advance,
-Hugo

Brian Jackson

2013-01-29 21:37 UTC #1 lore raw

Quoted 78 lines · On Tue, 29 Jan 2013 12:53:21 -0500 Hugo R Hernández-Mora <[email protected]>…

Hello there,
we are experiencing a problem by configuring a KVM bridged networking to share a public network interface between the KVM host and the VMs. Currently, our KVM server has set three network interfaces as follows:
* eth0: 192.168.10.101/23 (main interface for public network - no bridge)
* eth1 <--> br1: 192.168.10.201/23 (KVM VMs connected to public network)
* eth3 <--> br3: 10.7.10.201/23 (KVM VMs connected to LAN)
We have followed instructions as from Red Hat as well as from diferrent web sites and we are not able to get the VMs to get access into/from the public network. Here is a more detailed configuration for the KVM host:
ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
HWADDR=AC:80:B2:14:C5:EE
BOOTPROTO=none
IPADDR=192.168.10.101
NETMASK=255.255.254.0
ifcfg-eth1
DEVICE=eth1
ONBOOT=yes
HWADDR=AC:80:B2:4E:D3:28
BRIDGE=br1
ifcfg-br1
DEVICE=br1
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=none
IPADDR=192.168.10.201
NETMASK=255.255.254.0
STP=off
DELAY=0
ifcfg-eth3
DEVICE=eth3
ONBOOT=yes
HWADDR=AC:80:B2:4E:D3:2A
BRIDGE=br3
ifcfg-br3
DEVICE=br3
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=static
IPADDR=10.7.10.201
NETMASK=255.255.254.0
STP=off
DELAY=0
network
NETWORKING=yes
HOSTNAME=kvm1.public-lan.net
GATEWAY=192.168.10.1
For iptables/routing, we have followed instructions as explained on
http://www.linux-kvm.org/page/Networking#public_bridge
*nat
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING --out-interface br1 -j MASQUERADE
COMMIT
:FORWARD ACCEPT [0:0]
-A FORWARD --in-interface br1 -j ACCEPT
Hostside:
Allow IPv4 forwarding and add route to client (could be put in a script
- route has to be added after the client has started):
sysctl -w net.ipv4.ip_forward=1 # allow forwarding of IPv4 route add -host <ip-of-client> dev <tap-device> # add route to the client

Quoted 4 lines

Clientside:
Default GW of the client is of course then the host (<ip-of-host> has to be in same subnet as <ip-of-client> ...):
route add default gw <ip-of-host>

What do the client configs look like? What network options are you passing to qemu/kvm (or just the whole command line)? If your guests and host are in the same subnet, why are you masquerading/routing? Why not just use standard bridging?

Quoted 15 lines

But it doesn't seem to work. My assumption the problem is related with a wrong setting of the firewall on the iptables. Could you please advice? Your help will be greatly appreciated!
We are running Scientific Linux 6.2 on the KVM server as well as on the VMs. There is no network issue by accessing the LAN between VMs but only to face the public network.
Thanks in advance,
-Hugo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hugo R. Hernandez-Mora

author

2013-01-29 22:15 UTC #2 lore raw

Brian,
thanks for having the time and look into my problem. I have set my
VMs by using virt-manager but here is how it looks the qemu/kvm process running for my client:

[root@kvm1 ~]# ps -efl | grep qemu
6 S qemu 3532 1 1 80 0 - 2834530 poll_s 11:38 ?
00:03:20 /usr/libexec/qemu-kvm -S -M rhel6.3.0 -enable-kvm -m 8192
-smp 2,sockets=2,cores=1,threads=1 -name jacobi -uuid 740569a2-613f-ee1b-14fd-02772e28b211 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/jacobi.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
-no-shutdown -boot order=cd,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/ifs/virt/vm3/jacobi.img,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:ea:44:67,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0
-vnc 127.0.0.1:0 -vga cirrus -device
intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -incoming fd:22 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

I'm using a standard way for setting up networking as assigning a static IP for iface eth0 52:54:00:ea:44:67. I have changed my firewall rules to use only this as from documentation and by having in mind what you said about having the VM on same network as the KVM host:

iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT

I'm not sure if the problem is because a port blocking on the network
switch or a misconfiguration from my side. Anyways, I have tried to
route VM by using the same default gateway used by the KVM host, or to use the KVM host as gateway but any of these two options work in my case.

Thoughts?

Regards,
-Hugo

Signature · 6 lines

Hugo R Hernández-Mora

"Se seus esforços foram vistos com indeferença, não desanime que o sol
faze um espectacolo maravilhoso todas as manhãs enquanto a maioria das
pessoas ainda estão dormindo"
- Anónimo brasileiro

Quoted 109 lines · Brian Jackson

Quoted 78 lines · On Tue, 29 Jan 2013 12:53:21 -0500 Hugo R Hernández-Mora <[email protected]>…
Hello there,
we are experiencing a problem by configuring a KVM bridged networking to share a public network interface between the KVM host and the VMs. Currently, our KVM server has set three network interfaces as follows:
* eth0: 192.168.10.101/23 (main interface for public network - no bridge)
* eth1 <--> br1: 192.168.10.201/23 (KVM VMs connected to public network)
* eth3 <--> br3: 10.7.10.201/23 (KVM VMs connected to LAN)
We have followed instructions as from Red Hat as well as from diferrent web sites and we are not able to get the VMs to get access into/from the public network. Here is a more detailed configuration for the KVM host:
ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
HWADDR=AC:80:B2:14:C5:EE
BOOTPROTO=none
IPADDR=192.168.10.101
NETMASK=255.255.254.0
ifcfg-eth1
DEVICE=eth1
ONBOOT=yes
HWADDR=AC:80:B2:4E:D3:28
BRIDGE=br1
ifcfg-br1
DEVICE=br1
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=none
IPADDR=192.168.10.201
NETMASK=255.255.254.0
STP=off
DELAY=0
ifcfg-eth3
DEVICE=eth3
ONBOOT=yes
HWADDR=AC:80:B2:4E:D3:2A
BRIDGE=br3
ifcfg-br3
DEVICE=br3
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=static
IPADDR=10.7.10.201
NETMASK=255.255.254.0
STP=off
DELAY=0
network
NETWORKING=yes
HOSTNAME=kvm1.public-lan.net
GATEWAY=192.168.10.1
For iptables/routing, we have followed instructions as explained on
http://www.linux-kvm.org/page/Networking#public_bridge
*nat
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING --out-interface br1 -j MASQUERADE
COMMIT
:FORWARD ACCEPT [0:0]
-A FORWARD --in-interface br1 -j ACCEPT
Hostside:
Allow IPv4 forwarding and add route to client (could be put in a script
- route has to be added after the client has started):
sysctl -w net.ipv4.ip_forward=1 # allow forwarding of IPv4 route add -host <ip-of-client> dev <tap-device> # add route to the client
Quoted 4 lines
Clientside:
Default GW of the client is of course then the host (<ip-of-host> has to be in same subnet as <ip-of-client> ...):
route add default gw <ip-of-host>
What do the client configs look like? What network options are you passing to qemu/kvm (or just the whole command line)? If your guests and host are in the same subnet, why are you masquerading/routing? Why not just use standard bridging?
Quoted 15 lines
But it doesn't seem to work. My assumption the problem is related with a wrong setting of the firewall on the iptables. Could you please advice? Your help will be greatly appreciated!
We are running Scientific Linux 6.2 on the KVM server as well as on the VMs. There is no network issue by accessing the LAN between VMs but only to face the public network.
Thanks in advance,
-Hugo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Brian Jackson

2013-01-29 23:23 UTC #3 lore raw

Quoted 19 lines · On Tue, 29 Jan 2013 17:15:54 -0500 "Hugo R. Hernandez-Mora"…

Brian,
thanks for having the time and look into my problem. I have set my
VMs by using virt-manager but here is how it looks the qemu/kvm process running for my client:
[root@kvm1 ~]# ps -efl | grep qemu
6 S qemu 3532 1 1 80 0 - 2834530 poll_s 11:38 ?
00:03:20 /usr/libexec/qemu-kvm -S -M rhel6.3.0 -enable-kvm -m 8192 -smp 2,sockets=2,cores=1,threads=1 -name jacobi -uuid 740569a2-613f-ee1b-14fd-02772e28b211 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/jacobi.monitor,server,nowait
(quoted diff) +0 -2
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
-no-shutdown -boot order=cd,menu=on -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/ifs/virt/vm3/jacobi.img,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0

Quoted 8 lines

-netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:ea:44:67,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -incoming fd:22 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

Unfortunately, from that it's hard to tell what's actually connected to what. Curse libvirt for that.

Quoted 7 lines

I'm using a standard way for setting up networking as assigning a static IP for iface eth0 52:54:00:ea:44:67. I have changed my firewall rules to use only this as from documentation and by having in mind what you said about having the VM on same network as the KVM host:

Quoted 7 lines

iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
I'm not sure if the problem is because a port blocking on the network
switch or a misconfiguration from my side. Anyways, I have tried to
route VM by using the same default gateway used by the KVM host, or to use the KVM host as gateway but any of these two options work in my case.

A "normal" bridge setup wouldn't require any iptables rules to work, so why don't you try disabling all your iptables rules on the host and guest and setting the guest to use the same router as the host. See what that gets you. Try pinging and tcpdumping at different points to see where exactly things are failing.

Quoted 5 lines

Thoughts?
Regards,
-Hugo