Problems with KVM-79
Hi,
I was running KVM-76 with 2 CentOS 5.2 64bit VMs until a few days ago. Everything was working fine. However, I needed to install a third VM running Windows 2008 Server 64bit, which randomly kept crashing the host system with a kernel panic, so I thought, why not upgrade to KVM-79, wanted to do that for a while anyways. Now, Windows 2008 Server is running absolutely fine, no more host crashes. However, the two CentOS VMs crash on me every once in a while for no good reason. This apparently seems to happen more often when one of the CentOS VMs is under load, like compiling something. I had one of the VMs crash on me like 10 times to get a PHP 5.2.7 compile thru. I tried starting up the crashing VM with -no-kvm, and the problem at least seemed to go away. I didn't leave it running like this too long as the VM is in production and with -no-kvm, everything ran terribly slow.
Here's some infos about the system I'm using:
Hardware:
- Intel Core2Quad Q6600
- 8 GB RAM (checked with memtest86+ to make sure it's good)
- 2x 320 GB Seagate drives in a software RAID-1
Software:
- Host system: CentOS 5.2 64bit
- VMs: 2x CentOS 5.2 64bit (same kernel as host, see below), 1x Windows Server 2008 64bit, all available updates installed
- manually compiled KVM-79
- Host kernel version: 2.6.18-92.1.18.el5
qemu commands:
VM1 (CentOS 5.2 64bit):
export DISPLAY=:1
qemu-kvm -daemonize -localtime -no-acpi -hda /kvm/vm1.img -m 1024 -boot c -net nic,macaddr=52:54:00:00:01:00,model=e1000 -net tap,ifname=tap0,script=no
VM2 (Windows Server 2008 64bit):
export DISPLAY=:2
qemu-kvm -daemonize -localtime -hda /kvm/vm2.img -m 2048 -boot c -net nic,macaddr=52:54:00:00:01:01,model=e1000 -net tap,ifname=tap1,script=no
VM3 (CentOS 5.2 64bit):
export DISPLAY=:3
qemu-kvm -daemonize -localtime -hda /kvm/vm3.img -m 1024 -boot c -net nic,macaddr=52:54:00:00:01:02,model=e1000 -net tap,ifname=tap2,script=no
(qemu-kvm is a symlink to qemu-system-x86_64, inspired by the CentOS packages provided by lfarkas.org. Also tried those instead of compiling manually, btw - same problem.)
When the crashes happen, the qemu processes are still running on the host, but consume a lot of CPU. Checking the vnc console, there is no output whatsoever from the VMs, just a blank, black screen that doesn't react to anything (even tried SysRq, nothing).
Also, what I noticed is that CentOS 5.2 spits loads of errors about kernel modules (don't have the exact message ready now, but I could get it later on if needed) when booting up as soon as -smp >=2 is used. This also did not happen with KVM-76. When I run with -smp 1 or leave that parameter out, everything works well - except for the random crashes.
As this did in no way happen with KVM-76, I figure there must me a bug in the most recent version causing this issue.
I'd appreciate any help you could provide. If there's anything else I can do, like provide debugging output etc., please just let me know.
Signature · 9 lines
Mit freundlichen Gruessen / Best regards Chris Kuhles Geschaeftsleitung / Managing Director Aquatix IT-Services e.K. Telefon: +49 69 24750456-0 Münzgasse 6 Telefax: +49 69 24750456-1 60311 Frankfurt am Main E-Mail: [email protected] AG Frankfurt am Main, HRA 44038 Web: http://www.aquatix.de/
Hi,
in addition to this, I just tried updating to the most recent development code (got it via git). The CentOS 5.2 VMs at least booted up fine, however, the Windows 2008 Server immediately threw a bluescreen right after asking whether to boot regularly or in safe mode (just killed of the qemu process before). Switching back to KVM-79 again, everything ran "fine" again, except for the problems explained below. So this is just to let you know that Win2008 support seems broken in the current development tree.
Cheers,
Chris
Quoted 67 lines
Hi,
I was running KVM-76 with 2 CentOS 5.2 64bit VMs until a few days ago. Everything was working fine. However, I needed to install a third VM running Windows 2008 Server 64bit, which randomly kept crashing the host system with a kernel panic, so I thought, why not upgrade to KVM-79, wanted to do that for a while anyways. Now, Windows 2008 Server is running absolutely fine, no more host crashes. However, the two CentOS VMs crash on me every once in a while for no good reason. This apparently seems to happen more often when one of the CentOS VMs is under load, like compiling something. I had one of the VMs crash on me like 10 times to get a PHP 5.2.7 compile thru. I tried starting up the crashing VM with -no-kvm, and the problem at least seemed to go away. I didn't leave it running like this too long as the VM is in production and with -no-kvm, everything ran terribly slow.
Here's some infos about the system I'm using:
Hardware:
- Intel Core2Quad Q6600
- 8 GB RAM (checked with memtest86+ to make sure it's good)
- 2x 320 GB Seagate drives in a software RAID-1Software:
- Host system: CentOS 5.2 64bit
- VMs: 2x CentOS 5.2 64bit (same kernel as host, see below), 1x Windows Server 2008 64bit, all available updates installed
- manually compiled KVM-79
- Host kernel version: 2.6.18-92.1.18.el5qemu commands:
VM1 (CentOS 5.2 64bit):
export DISPLAY=:1
qemu-kvm -daemonize -localtime -no-acpi -hda /kvm/vm1.img -m 1024 -boot c -net nic,macaddr=52:54:00:00:01:00,model=e1000 -net tap,ifname=tap0,script=noVM2 (Windows Server 2008 64bit):
export DISPLAY=:2
qemu-kvm -daemonize -localtime -hda /kvm/vm2.img -m 2048 -boot c -net nic,macaddr=52:54:00:00:01:01,model=e1000 -net tap,ifname=tap1,script=noVM3 (CentOS 5.2 64bit):
export DISPLAY=:3
qemu-kvm -daemonize -localtime -hda /kvm/vm3.img -m 1024 -boot c -net nic,macaddr=52:54:00:00:01:02,model=e1000 -net tap,ifname=tap2,script=no(qemu-kvm is a symlink to qemu-system-x86_64, inspired by the CentOS packages provided by lfarkas.org. Also tried those instead of compiling manually, btw - same problem.)
When the crashes happen, the qemu processes are still running on the host, but consume a lot of CPU. Checking the vnc console, there is no output whatsoever from the VMs, just a blank, black screen that doesn't react to anything (even tried SysRq, nothing).
Also, what I noticed is that CentOS 5.2 spits loads of errors about kernel modules (don't have the exact message ready now, but I could get it later on if needed) when booting up as soon as -smp >=2 is used. This also did not happen with KVM-76. When I run with -smp 1 or leave that parameter out, everything works well - except for the random crashes.
As this did in no way happen with KVM-76, I figure there must me a bug in the most recent version causing this issue.
I'd appreciate any help you could provide. If there's anything else I can do, like provide debugging output etc., please just let me know.
Signature · 9 lines
Mit freundlichen Gruessen / Best regards Chris Kuhles Geschaeftsleitung / Managing Director Aquatix IT-Services e.K. Telefon: +49 69 24750456-0 Münzgasse 6 Telefax: +49 69 24750456-1 60311 Frankfurt am Main E-Mail: [email protected] AG Frankfurt am Main, HRA 44038 Web: http://www.aquatix.de/