New: Migration problem of qemu-kvm guests

https://bugzilla.kernel.org/show_bug.cgi?id=61971

            Bug ID: 61971
           Summary: Migration problem of qemu-kvm guests
           Product: Virtualization
           Version: unspecified
    Kernel Version: 3.10.5
          Hardware: x86-64
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: kvm
          Assignee: [email protected]
          Reporter: [email protected]
        Regression: No

Hello,

I am having migration problems and would appreciate very much help in troubleshooting this further.

I have installed two wheezy hosts (qemu-1.1.2) using glusterfs as backend storage and are running about 12 debian wheezy guests.

Node A has newer hardware (AMD FX(tm)-8320 Eight-Core Processor) and node B is older (AMD Phenom(tm) II X6 1055T Processor).

I am using libvirt and when migrating hosts from Node A to node B the always freeze indicating high cpu utilization in virt-manager for example. Migrations from Node B to Node A always works. I can start all gests on both hosts without problems.

To rule out glusterfs as the problem I have also used nfs as backend storage with the same results. I have installed a freebsd-9.1 guest to rule out the guests linux kernel as the likely problem. The guests also freeze if they are in grub or the freebsd bootloader while being migrated.

I have tried using this kernel parameter processor.max_cstate=1 and using changing clocksource to acpi_pm (defaults to kvm-clock). Neither of these had any effect on my problem.

I have tried disabling the balloon driver and that didn't help either.

Both hosts and all guests use ntp so the time is synced.

At one point I reinstalled both machines as ubuntu 13.04 to get newer versions of qemu (1.4) and libvirt but the exact same problem was there also, as I prefer debian I went back to wheezy.

-no-kvm-irqchip and -no-kvm-pit doesn't help either.

Using -no-kvm the migration works both ways.

My gut feeling is that it has something to do with the different cpu's, but I read that it was possible to migrate between intel and amd I guess this shouldn't be a problem.

Anyone have any ide what could be the problem here?

Best Regards
Magnus

Diff between cpuinfo of linux guest when booted on each host so no cpu flags differ in the guest. I have tried with Opteron_G2, qemu64 and kvm64 cpus all causes the guest to freeze.

root@lazy:~# diff -Nru iron1-cpuinfo.txt iron2-cpuinfo.txt
--- iron1-cpuinfo.txt   2013-09-19 20:59:24.407659093 +0200
+++ iron2-cpuinfo.txt   2013-09-19 21:01:03.326022357 +0200
@@ -5,14 +5,14 @@
 model name     : QEMU Virtual CPU version 1.1.2
 stepping       : 3
 microcode      : 0x1000065
-cpu MHz                : 3511.722
+cpu MHz                : 2812.460
 cache size     : 512 KB
 fpu            : yes
 fpu_exception  : yes
 cpuid level    : 4
 wp             : yes

flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm up nopl pni cx16 popcnt hypervisor lahf_lm svm abm sse4a

-bogomips       : 7023.44
+bogomips       : 5624.92
 TLB size       : 1024 4K pages
 clflush size   : 64

cache_alignment : 64

Node A cpuinfo
processor : 7
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8320 Eight-Core Processor
stepping : 0
microcode : 0x600081f
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 7
cpu cores : 4
apicid : 23
initial apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1

bogomips : 7023.38
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

Node B cpuinfo
processor : 5
vendor_id : AuthenticAMD
cpu family : 16
model : 10
model name : AMD Phenom(tm) II X6 1055T Processor
stepping : 0
microcode : 0x10000dc
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 6
core id : 5
cpu cores : 6
apicid : 5
initial apicid : 5
fpu : yes
fpu_exception : yes
cpuid level : 6
wp : yes

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter

bogomips : 5624.50
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate cpb

Libvirt starts the guests like this:

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-1.1 -enable-kvm -m 512 -smp 2,sockets=2,cores=1,threads=1 -name jefster -uuid f94a85d4-629e-bdba-b585-282638526800 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/jefster.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/vmimages/gv0/jefster.img,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:61:9b:f3,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:5 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

Hosts kernel:
Linux iron1 3.10-0.bpo.2-amd64 #1 SMP Debian 3.10.5-1~bpo70+1 (2013-08-11) x86_64 GNU/Linux

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

Magnus Boden <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

     Kernel Version|3.10.5                      |3.11.1

--- Comment #1 from Magnus Boden <[email protected]> --- Hello,

Since the kernel I used was a debian backport I compiled my own kernel from kernel.org 3.11.1 and it has the same problem.

Regards
Magnus

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #2 from Magnus Boden <[email protected]> --- Hello,

For testing I took an old machine and installed it like the other two except no need for local storage so no glusterfs. gluster volumes are mounted from the other servers.

Migration
Node A -> (B OR C) Guest freeze
Node (B OR C) -> Node A works
Node B <-> C Works

Any debugging or troubleshooting steps I can do I am happy for any pointers in the right direction.

Regards
Magnus

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #3 from Magnus Boden <[email protected]> --- Can anyone help with troubleshooting this further?

Regards
Magnus

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

Gleb <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #4 from Gleb <[email protected]> ---
So new to old always hangs? What "info status" shows when guest freeze? Can you try latest qemu?

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #5 from Magnus Boden <[email protected]> --- Hello,

It says this, I haven't found any easy way to upgrade to later versions of qemu in debian but I will try compiling it from source.

I did try 1.4.0 and 1.5.0 in ubuntu before reverting back to my prefered distro debian.

virsh # qemu-monitor-command --hmp lazy "info status" VM status: running

Thanks
Magnus

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #6 from Magnus Boden <[email protected]> --- Sorry missed to answer your question:

So new to old always hangs?

Yes it always hangs.

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

Paolo Bonzini <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #7 from Paolo Bonzini <[email protected]> --- Can you copy here the cpuinfo of the two hosts?

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #8 from Magnus Boden <[email protected]> ---
Node A
processor : 0
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8320 Eight-Core Processor
stepping : 0
microcode : 0x600081f
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 0
cpu cores : 4
apicid : 16
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 7023.50
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

processor : 1
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8320 Eight-Core Processor
stepping : 0
microcode : 0x600081f
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 1
cpu cores : 4
apicid : 17
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 7023.50
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

processor : 2
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8320 Eight-Core Processor
stepping : 0
microcode : 0x600081f
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 2
cpu cores : 4
apicid : 18
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 7023.50
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

processor : 3
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8320 Eight-Core Processor
stepping : 0
microcode : 0x600081f
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 3
cpu cores : 4
apicid : 19
initial apicid : 3
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 7023.50
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

processor : 4
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8320 Eight-Core Processor
stepping : 0
microcode : 0x600081f
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 4
cpu cores : 4
apicid : 20
initial apicid : 4
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 7023.50
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

processor : 5
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8320 Eight-Core Processor
stepping : 0
microcode : 0x600081f
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 5
cpu cores : 4
apicid : 21
initial apicid : 5
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 7023.50
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

processor : 6
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8320 Eight-Core Processor
stepping : 0
microcode : 0x600081f
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 6
cpu cores : 4
apicid : 22
initial apicid : 6
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 7023.50
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

processor : 7
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD FX(tm)-8320 Eight-Core Processor
stepping : 0
microcode : 0x600081f
cpu MHz : 1400.000
cache size : 2048 KB
physical id : 0
siblings : 8
core id : 7
cpu cores : 4
apicid : 23
initial apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 7023.50
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #9 from Magnus Boden <[email protected]> ---
Node B
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 10
model name : AMD Phenom(tm) II X6 1055T Processor
stepping : 0
microcode : 0x10000dc
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 6
core id : 0
cpu cores : 6
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 6
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter
bogomips : 5625.25
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate cpb

processor : 1
vendor_id : AuthenticAMD
cpu family : 16
model : 10
model name : AMD Phenom(tm) II X6 1055T Processor
stepping : 0
microcode : 0x10000dc
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 6
core id : 1
cpu cores : 6
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 6
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter
bogomips : 5625.25
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate cpb

processor : 2
vendor_id : AuthenticAMD
cpu family : 16
model : 10
model name : AMD Phenom(tm) II X6 1055T Processor
stepping : 0
microcode : 0x10000dc
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 6
core id : 2
cpu cores : 6
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 6
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter
bogomips : 5625.25
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate cpb

processor : 3
vendor_id : AuthenticAMD
cpu family : 16
model : 10
model name : AMD Phenom(tm) II X6 1055T Processor
stepping : 0
microcode : 0x10000dc
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 6
core id : 3
cpu cores : 6
apicid : 3
initial apicid : 3
fpu : yes
fpu_exception : yes
cpuid level : 6
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter
bogomips : 5625.25
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate cpb

processor : 4
vendor_id : AuthenticAMD
cpu family : 16
model : 10
model name : AMD Phenom(tm) II X6 1055T Processor
stepping : 0
microcode : 0x10000dc
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 6
core id : 4
cpu cores : 6
apicid : 4
initial apicid : 4
fpu : yes
fpu_exception : yes
cpuid level : 6
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter
bogomips : 5625.25
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate cpb

processor : 5
vendor_id : AuthenticAMD
cpu family : 16
model : 10
model name : AMD Phenom(tm) II X6 1055T Processor
stepping : 0
microcode : 0x10000dc
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 6
core id : 5
cpu cores : 6
apicid : 5
initial apicid : 5
fpu : yes
fpu_exception : yes
cpuid level : 6
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter
bogomips : 5625.25
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate cpb

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #10 from Magnus Boden <[email protected]> ---
Node C
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : AMD Phenom(tm) 8550 Triple-Core Processor
stepping : 3
microcode : 0x1000065
cpu MHz : 1100.000
cache size : 512 KB
physical id : 0
siblings : 3
core id : 0
cpu cores : 3
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs hw_pstate npt lbrv svm_lock
bogomips : 4420.10
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor : 1
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : AMD Phenom(tm) 8550 Triple-Core Processor
stepping : 3
microcode : 0x1000065
cpu MHz : 1100.000
cache size : 512 KB
physical id : 0
siblings : 3
core id : 1
cpu cores : 3
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs hw_pstate npt lbrv svm_lock
bogomips : 4420.10
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor : 2
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : AMD Phenom(tm) 8550 Triple-Core Processor
stepping : 3
microcode : 0x1000065
cpu MHz : 1100.000
cache size : 512 KB
physical id : 0
siblings : 3
core id : 2
cpu cores : 3
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs hw_pstate npt lbrv svm_lock
bogomips : 4420.10
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #11 from Paolo Bonzini <[email protected]> ---
This is XSAVE -> non-XSAVE migration. Alexander Graf reported to me on IRC that the XSAVE patches we recently committed to kvm/next fixed this for him. xstate_bv is saved as 7 and this check fails on the destination:

if (xstate_bv & ~XSTATE_FPSSE)
return -EINVAL;

I think it is because your CPU has xsave but not xsaveopt.

However, those patches introduced regressions that have to be fixed. I'll look at those soon, but your bug should get fixed in 3.13.

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #12 from Magnus Boden <[email protected]> --- Perfect. Thanks for the response.

I will try 3.13 as soon as it is released.

If you need any beta testing I am available.

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #13 from Gleb <[email protected]> ---
(In reply to Magnus Boden from comment #12)

Can you try current kvm.git next branch? The bugs that is there may not affect your case.

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #14 from Magnus Boden <[email protected]> ---
I will try that. I am going away tonight but I will try it tomorrow and report back tomorrow evening.

Thanks for the support
Magnus

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #15 from Magnus Boden <[email protected]> --- I have no upgraded all three hosts with the kernel from
git://git.kernel.org/pub/scm/virt/kvm/kvm.git

Using the branch next, last commit on that tree was:
commit d570142674890fe10b3d7d86aa105e3dfce1ddfa
Merge: f2e1066 a7265fb
Author: Gleb Natapov <[email protected]>
Date: Wed Oct 16 15:30:32 2013 +0300

    Merge tag 'kvm-arm-for-3.13-1' of

git://git.linaro.org/people/cdall/linux-kvm-arm into next

Updates for KVM/ARM including cpu=host and Cortex-A7 support

Now the symptoms are the other way around.

Migrations B or C to A guests freeze (both freebsd and linux tested). Migrations A to B or C works.

Regards
Magnus

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #16 from Paolo Bonzini <[email protected]> ---
Yes, that's expected. You also need the patch I send on Thursday, that you were CCed on (KVM: x86: fix KVM_SET_XCRS for CPUs that do not support XSAVE).

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

--- Comment #17 from Magnus Boden <[email protected]> --- Sorry I missed the patch.

I have patched my kernel now and can report that migrations work in every direction between my three hosts.

Thanks very much for the support
Magnus

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=61971

Magnus Boden <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
         Resolution|---                         |PATCH_ALREADY_AVAILABLE

You are receiving this mail because:
You are watching the assignee of the bug.