stability issue with KVM using SMP
Summary - I'm getting stability issues running SMP. The guest will run for, typically, 6 to 12 hours before causing a problem.
The first time it happened the guest process that was running SMP went Zombie, but in a spin loop clocking huge amount of CPU time, and I had to reboot the host to clear it. Needless to say I had no access to the guest's qmeu console (Ctrl-Alt-2) and a "kill -9 <pid>" from the host had no effect.
However, the kernel had been compiled with a 64Gb memory model and I've had problems with that in the past - in fact that's the first time I've seen a kernel compiled with the 64Gb memory model actually boot under KVM.
So I replaced the kernel and the next time the guest simply locked up - I couldn't access the guest o/s at all (no "ping", no linux console response etc - in some cases there was a picture on the console [login prompt] but it didn't respond to any key presses), but the qemu console still worked. I tried a "system_powerdown" and that failed so I did a "system_reset" and that rebooted the client.
I have now switched off SMP and the guest is working fine (just SLOW).
I've had quite a few non-SMP linux guests running on this host for some time, using various kernels, and not seen a problem. Its only since I tried to introduce SMP that its all gone pear shaped.
Quoted 2 lines
# what cpu model (examples: Intel Core Duo, Intel Core 2 Duo, AMD Opteron 2210). See /proc/cpuinfo if you're not sure.
dual "Quad-Core AMD Opteron(tm) Processor 2352" with 32Gb RAM
Quoted 2 lines
# what kvm version you are using. If you're using git directly, provide the output of 'git describe'.
"kvm-72"
Quoted 1 line
# the host kernel version
"2.6.26.2" running on Slackware 12
Quoted 1 line
# what host kernel arch you are using (i386 or x86_64)
# CONFIG_64BIT is not setCONFIG_X86_32=y
# CONFIG_X86_64 is not setCONFIG_X86=y
Should I be using 64 bit ?
Quoted 2 lines
# what guest you are using, including OS type (Linux, Windows, Solaris, etc.), bitness (32 or 64), kernel version
Slackware 11, but with a new standard (no patches) kernel - 2.6.26.2
I also have an older Slackware 7 based guest with a 2.4.29 SMP kernel that sees the "lock-up, with qemu console working" problem, but not the Zombie issue. The 2.4.29 kernel uses a 1Gb memory model.
Quoted 1 line
# the qemu command line you are using to start the guest
/usr/local/bin/qemu-system-x86_64 -hda /opt/kvm/machine_14/vdisk1.img \
-m 1024 -vnc :14 -k en-gb -smp 4 \
-net nic,model=e1000,macaddr=52:54:00:14:00:00 -net tap \
-net nic,vlan=2,model=e1000,macaddr=52:54:00:14:00:01 \
-net tap,vlan=2,ifname=tap55 \Quoted 2 lines
# whether the problem goes away if using the -no-kvm-irqchip or -no-kvm-pit switch.
Not tried
Quoted 1 line
# whether the problem also appears with the -no-kvm switch.
I don't use this switch.
James
Further to my previous report..
I have just noticed that, where the host is running on the dual "Quad-Core AMD Opteron(tm) Processor 2352" (family 16, model 2) the kernels in the *guest* machines (usually 2.6.26) are reporting "WARNING: This combination of AMDprocessors is not suitable for SMP." - presumably this is why my SMP isn't stable!
The kernel in that host machine (2.6.26.2) does *not* report the messages.
However, where the host is a single "AMD Athlon(tm) 64 X2 Dual Core Processor 4000+" (family 15, model 107) the guest kernels do not report the WARNING above - but I've not actually run any SMP guest on that host, so don't know if it would work or not.
The warning seems to be due to a processor capability test in "arch/x86/kernel/smpboot.c" - "add_taint(TAINT_UNSAFE_SMP);"
Its not clear to me why one guess is failing this test and the other passing, because both guests are seeing "QEMU Virtual CPU version 0.9.1", family 6, model 2 (according to "/proc/cpuinfo").
Although most kernels on the Athlon host are running 2.6.21 not 2.6.26
James