qemu-kvm crashes doing migration with disks + blkdebug files (does not happen with qemu)
Hi folks,
qemu-kvm is segfaulting when executing migration with blkdebug files.
19:50:02 DEBUG| Git repo qemu_kvm uri: git://github.com/avikivity/qemu.git 19:50:02 DEBUG| Git repo qemu_kvm branch: master 19:50:30 INFO | Commit hash for qemu_kvm is 7879db7e9c09b92d9af1c143fbe2cc212ec89e4b (no tag found)
How to reproduce:
1) create a origin vm like:
/usr/local/autotest/tests/kvm/qemu -name 'vm1' -nodefaults -vga std -monitor unix:'/tmp/monitor-humanmonitor1-20111104-200902-95j0',server,nowait -qmp unix:'/tmp/monitor-qmpmonitor1-20111104-200902-95j0',server,nowait -serial unix:'/tmp/serial-20111104-200902-95j0',server,nowait -drive file=blkdebug:/usr/local/autotest/virt/blkdebug/default.conf:/tmp/kvm_autotest_root/images/rhel6.1-64.qcow2,index=0,if=virtio,cache=none,rerror=stop,werror=stop -device virtio-net-pci,netdev=idtzhBVb,mac='9a:d0:7b:07:18:72',id='id9JW3ZV' -netdev tap,id=idtzhBVb,fd=23 -m 2048 -smp 2 -vnc :0
2) create a destination vm like:
/usr/local/autotest/tests/kvm/qemu -name 'vm1' -nodefaults -vga std -monitor unix:'/tmp/monitor-humanmonitor1-20111104-201329-Ia9o',server,nowait -qmp unix:'/tmp/monitor-qmpmonitor1-20111104-201329-Ia9o',server,nowait -serial unix:'/tmp/serial-20111104-201329-Ia9o',server,nowait -drive file=blkdebug:/usr/local/autotest/virt/blkdebug/default.conf:/tmp/kvm_autotest_root/images/rhel6.1-64.qcow2,index=0,if=virtio,cache=none,rerror=stop,werror=stop -device virtio-net-pci,netdev=idup1xAf,mac='9a:d0:7b:07:18:72',id='idyvOQf3' -netdev tap,id=idup1xAf,fd=19 -m 2048 -smp 2 -vnc :1 -S -incoming "exec:nc -l 5200"
Note that blkdebug file contains:
[inject-error]
state = "2"
event = "read_aio"
errno = "7"
immediately = "off"
once = "on"
[set-state]
state = "1"
event = "read_aio"new_state = "2"
[set-state]
state = "2"
event = "read_aio"new_state = "3"
Start the migration (on this example, using exec, but it reproduces with tcp and unix sockets):
11/04 20:13:30 DEBUG|kvm_monito:0254| (monitor humanmonitor1) Sending command 'migrate -d "exec:nc localhost 5200"'
Then you will have:
11/04 20:13:33 INFO | aexpect:0783| [qemu output] invalid runstate transition 11/04 20:13:36 INFO | aexpect:0783| [qemu output] /bin/sh: line 1: 14695 Aborted (core dumped) /usr/local/autotest/tests/kvm/qemu -name 'vm1' -nodefaults -vga std -monitor unix:'/tmp/monitor-humanmonitor1-20111104-200902-95j0',server,nowait -qmp unix:'/tmp/monitor-qmpmonitor1-20111104-200902-95j0',server,nowait -serial unix:'/tmp/serial-20111104-200902-95j0',server,nowait -drive file=blkdebug:/usr/local/autotest/virt/blkdebug/default.conf:/tmp/kvm_autotest_root/images/rhel6.1-64.qcow2,index=0,if=virtio,cache=none,rerror=stop,werror=stop -device virtio-net-pci,netdev=idtzhBVb,mac='9a:d0:7b:07:18:72',id='id9JW3ZV' -netdev tap,id=idtzhBVb,fd=23 -m 2048 -smp 2 -vnc :0
We do have the core dumps available in case someone is interested in debugging the issue.
It is important to note that this problem is not happening with qemu.git master.
Am 05.11.2011 03:16, schrieb Lucas Meneghel Rodrigues:
Quoted 59 lines
Hi folks,
qemu-kvm is segfaulting when executing migration with blkdebug files.
19:50:02 DEBUG| Git repo qemu_kvm uri: git://github.com/avikivity/qemu.git 19:50:02 DEBUG| Git repo qemu_kvm branch: master 19:50:30 INFO | Commit hash for qemu_kvm is
7879db7e9c09b92d9af1c143fbe2cc212ec89e4b(no tag found)How to reproduce:
1) create a origin vm like:
/usr/local/autotest/tests/kvm/qemu-name 'vm1' -nodefaults -vga std -monitor unix:'/tmp/monitor-humanmonitor1-20111104-200902-95j0',server,nowait(quoted diff) +0
-2-qmp unix:'/tmp/monitor-qmpmonitor1-20111104-200902-95j0',server,nowait -serial unix:'/tmp/serial-20111104-200902-95j0',server,nowait -drivefile=blkdebug:/usr/local/autotest/virt/blkdebug/default.conf:/tmp/kvm_autotest_root/images/rhel6.1-64.qcow2,index=0,if=virtio,cache=none,rerror=stop,werror=stop -device
virtio-net-pci,netdev=idtzhBVb,mac='9a:d0:7b:07:18:72',id='id9JW3ZV'
-netdev tap,id=idtzhBVb,fd=23 -m 2048 -smp 2 -vnc :02) create a destination vm like:
/usr/local/autotest/tests/kvm/qemu-name 'vm1' -nodefaults -vga std -monitor unix:'/tmp/monitor-humanmonitor1-20111104-201329-Ia9o',server,nowait(quoted diff) +0
-2-qmp unix:'/tmp/monitor-qmpmonitor1-20111104-201329-Ia9o',server,nowait -serial unix:'/tmp/serial-20111104-201329-Ia9o',server,nowait -drivefile=blkdebug:/usr/local/autotest/virt/blkdebug/default.conf:/tmp/kvm_autotest_root/images/rhel6.1-64.qcow2,index=0,if=virtio,cache=none,rerror=stop,werror=stop -device
virtio-net-pci,netdev=idup1xAf,mac='9a:d0:7b:07:18:72',id='idyvOQf3'
-netdev tap,id=idup1xAf,fd=19 -m 2048 -smp 2 -vnc :1 -S -incoming
"exec:nc -l 5200"Note that blkdebug file contains:
[inject-error]
state = "2"
event = "read_aio"
errno = "7"
immediately = "off"
once = "on"[set-state]
state = "1"
event = "read_aio"new_state= "2"[set-state]
state = "2"
event = "read_aio"new_state= "3"Start the migration (on this example, using exec, but it reproduces with tcp and unix sockets):
Quoted 7 lines
11/04 20:13:30 DEBUG|kvm_monito:0254| (monitor humanmonitor1) Sending command 'migrate -d "exec:nc localhost 5200"'
Then you will have:
11/04 20:13:33 INFO | aexpect:0783| [qemu output] invalid runstate
transition
Invalid runstate transition is something for Luiz (CCed). Though probably he doesn't need to do anything in this case: I think we're not allowing the transition from I/O error to migrating. This might be fixed by 8a9236f1 in qemu.git, so please retest with upstream.
Kevin
Quoted 65 lines · Kevin Wolf
Am 05.11.2011 03:16, schrieb Lucas Meneghel Rodrigues:
Quoted 64 lines
Hi folks,
qemu-kvm is segfaulting when executing migration with blkdebug files.
19:50:02 DEBUG| Git repo qemu_kvm uri: git://github.com/avikivity/qemu.git 19:50:02 DEBUG| Git repo qemu_kvm branch: master 19:50:30 INFO | Commit hash for qemu_kvm is
7879db7e9c09b92d9af1c143fbe2cc212ec89e4b(no tag found)How to reproduce:
1) create a origin vm like:
/usr/local/autotest/tests/kvm/qemu-name 'vm1' -nodefaults -vga std -monitor unix:'/tmp/monitor-humanmonitor1-20111104-200902-95j0',server,nowait(quoted diff) +0
-2-qmp unix:'/tmp/monitor-qmpmonitor1-20111104-200902-95j0',server,nowait -serial unix:'/tmp/serial-20111104-200902-95j0',server,nowait -drivefile=blkdebug:/usr/local/autotest/virt/blkdebug/default.conf:/tmp/kvm_autotest_root/images/rhel6.1-64.qcow2,index=0,if=virtio,cache=none,rerror=stop,werror=stop -device
virtio-net-pci,netdev=idtzhBVb,mac='9a:d0:7b:07:18:72',id='id9JW3ZV'
-netdev tap,id=idtzhBVb,fd=23 -m 2048 -smp 2 -vnc :02) create a destination vm like:
/usr/local/autotest/tests/kvm/qemu-name 'vm1' -nodefaults -vga std -monitor unix:'/tmp/monitor-humanmonitor1-20111104-201329-Ia9o',server,nowait(quoted diff) +0
-2-qmp unix:'/tmp/monitor-qmpmonitor1-20111104-201329-Ia9o',server,nowait -serial unix:'/tmp/serial-20111104-201329-Ia9o',server,nowait -drivefile=blkdebug:/usr/local/autotest/virt/blkdebug/default.conf:/tmp/kvm_autotest_root/images/rhel6.1-64.qcow2,index=0,if=virtio,cache=none,rerror=stop,werror=stop -device
virtio-net-pci,netdev=idup1xAf,mac='9a:d0:7b:07:18:72',id='idyvOQf3'
-netdev tap,id=idup1xAf,fd=19 -m 2048 -smp 2 -vnc :1 -S -incoming
"exec:nc -l 5200"Note that blkdebug file contains:
[inject-error]
state = "2"
event = "read_aio"
errno = "7"
immediately = "off"
once = "on"[set-state]
state = "1"
event = "read_aio"new_state= "2"[set-state]
state = "2"
event = "read_aio"new_state= "3"Start the migration (on this example, using exec, but it reproduces with tcp and unix sockets):
11/04 20:13:30 DEBUG|kvm_monito:0254| (monitor humanmonitor1) Sending command 'migrate -d "exec:nc localhost 5200"'
Then you will have:
Quoted 7 lines
Quoted 2 lines
11/04 20:13:33 INFO | aexpect:0783| [qemu output] invalid runstate
transitionInvalid runstate transition is something for Luiz (CCed). Though probably he doesn't need to do anything in this case: I think we're not allowing the transition from I/O error to migrating. This might be fixed by
8a9236f1in qemu.git, so please retest with upstream.
In the end of my original message, I stated the same test does not yield a segfault in qemu.git, so the referred commit indeed fixes the issue.
Thanks!