New: KVM domain hangs after loading initrd with Xenomai kernel
https://bugzilla.kernel.org/show_bug.cgi?id=42779
Summary: KVM domain hangs after loading initrd with Xenomai
kernel
Product: Virtualization
Version: unspecified
Kernel Version: 3.0.0-15
Platform: All
OS/Version: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: kvm
AssignedTo: [email protected]
ReportedBy: [email protected]
Regression: No
Attempting to boot a 32 bit Debian guest with a Xenomai kernel inside KVM causes it to hang and spin (using 1 full CPU core) after loading the initrd, as determined by serial console output. The only error message is "KVM internal error. Suberror: 1"/"emulation failure". Booting a regular Debian kernel succeeds, as does running the Xenomai kernel with software emulation (-no-kvm).
Info:
CPU: Intel Core i7-2670QM
Emulator: qemu-kvm 0.14.1
Host kernel: 3.0.0-15 (Ubuntu build), x86_64
Guest OS: Debian Squeeze, kernel.org 2.6.37 kernel with Xenomai 2.6.0 (config attached)
Qemu command: kvm -M pc-0.14 -enable-kvm -m 1024 -drive file=/var/lib/libvirt/images/eve.img,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=21,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:b5:f4:00,bus=pci.0,addr=0x3 -chardev stdio,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0 -vga cirrus
Effects of flags: Adding one or both of --no-kvm-irqchip or --no-kvm-pit has no apparent effect. Adding --no-kvm appears to correct the problem, at the cost of performance due to using the software emulator.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #1 from [email protected] 2012-02-16 05:46:07 --- Created an attachment (id=72393)
--> (https://bugzilla.kernel.org/attachment.cgi?id=72393)
Configuration of the guest kernel
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #2 from [email protected] 2012-02-16 05:47:18 --- Created an attachment (id=72394)
--> (https://bugzilla.kernel.org/attachment.cgi?id=72394)
Result of 'registers info' and 'x/30i $eip' after fault
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
[email protected] changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #72393|application/octet-stream |text/plain
mime type| |Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
[email protected] changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #72394|application/octet-stream |text/plain
mime type| |Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #3 from [email protected] 2012-02-16 05:57:25 --- Couldn't attach the trace I recorded of the fault occurring since it's 3 MB compressed with xz, bigger still with other formats. I can email it if it will be useful.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #4 from [email protected] 2012-02-16 06:32:48 --- Same problem occurs with qemu-kvm 1.0 from
https://launchpad.net/~bderzhavets/+archive/lib-usbredir39:
$ sudo kvm -M pc-1.0 -enable-kvm -m 1024 -drive file=/var/lib/libvirt/images/eve.img,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=21,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:b5:f4:00,bus=pci.0,addr=0x3 -chardev vc,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0 -vga cirrus kvm: -netdev tap,fd=21,id=hostnet0: TUNGETIFF ioctl() failed: Bad file descriptor TUNSETOFFLOAD ioctl() failed: Bad file descriptor kvm: -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:b5:f4:00,bus=pci.0,addr=0x3: pci_add_option_rom: failed to find romfile "pxe-e1000.rom" KVM internal error. Suberror: 1 emulation failure EAX=f6810000 EBX=0000003e ECX=0000003e EDX=c00b8000 ESI=c00b8000 EDI=c15b0000 EBP=c15b1f74 ESP=c15b1f58 EIP=c1228905 EFL=00010206 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =007b 00000000 ffffffff 00c0f300 DPL=3 DS [-WA] CS =0060 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA] SS =0068 00000000 ffffffff 00c09300 DPL=0 DS [-WA] DS =007b 00000000 ffffffff 00c0f300 DPL=3 DS [-WA] FS =0000 00000000 ffffffff 00000000 GS =0000 00000000 ffffffff 00000000 LDT=0000 00000000 ffffffff 00000000 TR =0080 c15b6300 0000206b 00008b00 DPL=0 TSS32-busy GDT= c15b3000 000000ff IDT= c15b2000 000007ff CR0=80050033 CR2=ffee4000 CR3=01663000 CR4=00000690 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 Code=8e 2b 01 00 00 8b 4d f0 89 f2 8b 45 ec 0f 0d 82 40 01 00 00 <0f> 6f 02 0f 6f 4a 08 0f 6f 52 10 0f 6f 5a 18 0f 7f 00 0f 7f 48 08 0f 7f 50 10 0f 7f 58 18
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
Gleb <[email protected]> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]
--- Comment #5 from Gleb <[email protected]> 2012-02-16 07:15:49 --- (In reply to comment #3)
Quoted 3 lines
Couldn't attach the trace I recorded of the fault occurring since it's 3 MB compressed with xz, bigger still with other formats. I can email it if it will be useful.
Can you do "tail -10000" on it and attach it here?
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #6 from [email protected] 2012-02-16 07:22:41 --- Created an attachment (id=72395)
--> (https://bugzilla.kernel.org/attachment.cgi?id=72395)
Last 10k lines of a trace showing the fault
Per Gleb's request
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #7 from Gleb <[email protected]> 2012-02-16 07:43:11 --- Have you installed trace-cmd before capturing the trace? It failed to parse kvm events. qemu haven't paused the guest after emulation error (looks like a bug), so 'x/30i $eip' output is not useful either. Can you do 'x/30i 0xXXX' where XXX is the address in EIP from register dump you see after instruction emulation failure message (c1228905 in your output from comment #4)?
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #8 from [email protected] 2012-02-16 07:56:03 --- Not sure what you mean by installing trace-cmd before capturing the trace--I did do that, otherwise I wouldn't have had a trace-cmd to run. The package version is trace-cmd 1.0.3-0ubuntu1 if that helps. I tried running it again against qemu 1.0 (the last one was for qemu 0.14), still contained a bunch of [FAILED TO PARSE] messages.
Attaching the output you requested separately.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #9 from [email protected] 2012-02-16 07:57:35 --- Created an attachment (id=72397)
--> (https://bugzilla.kernel.org/attachment.cgi?id=72397)
Register state and code disassembly at failure point with qemu 1.0
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #10 from Gleb <[email protected]> 2012-02-16 08:04:22 --- (In reply to comment #8)
Quoted 2 lines
Not sure what you mean by installing trace-cmd before capturing the trace--I did do that, otherwise I wouldn't have had a trace-cmd to run. The package
I meant that if you compile it by yourself you need to run "make install" instead of running it from where it was compiled. Otherwise it does not find plugins. If you install it from package manager this is not relevant of course.
Quoted 4 lines
version is trace-cmd 1.0.3-0ubuntu1 if that helps. I tried running it again against qemu 1.0 (the last one was for qemu 0.14), still contained a bunch of [FAILED TO PARSE] messages.
Something wrong with ubuntu trace-cmd then. Either it does not have kvm plugin or it is too old.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
Jan Kiszka <[email protected]> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]
--- Comment #11 from Jan Kiszka <[email protected]> 2012-02-16 08:31:18 --- (In reply to comment #4)
Quoted 2 lines
Code=8e 2b 01 00 00 8b 4d f0 89 f2 8b 45 ec 0f 0d 82 40 01 00 00 <0f> 6f 02 0f 6f 4a 08 0f 6f 52 10 0f 6f 5a 18 0f 7f 00 0f 7f 48 08 0f 7f 50 10 0f 7f 58 18
This is "movq (%rdx),%mm0". Possibly still unsupported by the emulator.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #12 from Gleb <[email protected]> 2012-02-16 08:40:37 --- Can you try disabling CONFIG_FB in your guest kernel?
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
Avi Kivity <[email protected]> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]
--- Comment #13 from Avi Kivity <[email protected]> 2012-02-16 11:40:22 --- Indeed the emulator doesn't support mmx yet. But why is the kernel using mmx instead of sse?
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #14 from Jan Kiszka <[email protected]> 2012-02-16 11:51:05 --- Because the selected CPU type doesn't support SSE? Didn't check the config yet, but I bet that's the reason.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #15 from [email protected] 2012-02-16 16:48:56 --- I set the kernel up for an AMD Geode LX800 (CONFIG_MGEODE_LX), which does supposedly support SSE. No idea why it would have built using MMX if that is now deprecated in favor of SSE.
I can try disabling the framebuffer (CONFIG_FB/CONFIG_FB_GEODE/CONFIG_FB_GEODE_LX) for testing purposes; unfortunately this wouldn't work for the real thing, since we need it for the Qt-based user interface.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #16 from [email protected] 2012-02-16 16:51:10 ---
Background: I'm using KVM as a test platform for code destined for a panel PC using a Geode LX800.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #17 from Jan Kiszka <[email protected]> 2012-02-16 17:24:34 --- (In reply to comment #15)
Quoted 3 lines
I set the kernel up for an AMD Geode LX800 (
CONFIG_MGEODE_LX), which does supposedly support SSE. No idea why it would have built using MMX if that is now deprecated in favor of SSE.
At least not according to the Linux configuration: MGEODE_LX only sets X86_USE_3DNOW, thus MMX.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #18 from Avi Kivity <[email protected]> 2012-02-16 17:30:47 --- As a workaround you can use -cpu blah,-3dnow. But we'll have to implement mmx movq.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #19 from [email protected] 2012-02-16 22:01:38 --- Doesn't seem to help. I used -cpu core2duo,-3dnow (since core2duo is the name that libvirt, my usual launcher for qemu, has been using), failed on the same movq. Also tried -cpu core2duo,-3dnow,-mmx, same results. I'll see if I can build a kernel with the Geode framebuffer driver but without setting the CPU to Geode (thus eliminating the use of MMX).
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #20 from Avi Kivity <[email protected]> 2012-03-21 18:21:33 --- Please try
git://git.kernel.org/pub/scm/virt/kvm/kvm.git emulator-movq
this emulates the two movq instructions used. There may be other unemulated instructions though.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #21 from [email protected] 2012-03-31 20:10:38 --- Sorry for the delay, I've been really busy lately. Is this kernel reasonably safe to run on my main machine? I'm not worried about it crashing, but if it takes a filesystem with it...
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #22 from Avi Kivity <[email protected]> 2012-04-01 14:15:51 --- No guarantees of course, but this is just 3.3-rc5 with a few patches on top. It should be fine for a short test.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #23 from [email protected] 2012-04-01 23:38:21 --- Now it dies in a different place, still with a movq instruction (movq (%edx),%mm0). I think the difference is because of a configuration change or package update since the last test, because the fault happens at the same address and same instruction under both the patched 3.3rc5 and 3.0.20. Unfortunately I've no idea what the configuration change might have been, and I didn't think to save a VM disk image from the previous tests.
I also got a more recent trace-cmd built and installed, so there is usable trace output now. Still huge, so I'll post the last 10k lines as before.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #24 from [email protected] 2012-04-01 23:39:47 --- Created an attachment (id=72788)
--> (https://bugzilla.kernel.org/attachment.cgi?id=72788)
Register state and code disassembly at failure point, kernel 3.3rc5 with patches
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #25 from [email protected] 2012-04-01 23:40:21 --- Created an attachment (id=72789)
--> (https://bugzilla.kernel.org/attachment.cgi?id=72789)
Last 10k lines of a trace showing the fault, kernel 3.3rc5 with patches
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
--- Comment #26 from Avi Kivity <[email protected]> 2012-04-02 09:12:38 --- Strange, looks like the patches did not take effect at all. The opcode is 0f 6f 02, which should have been decoded as movq.
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
Florian Mickler <[email protected]> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]
--- Comment #27 from Florian Mickler <[email protected]> 2012-07-01 09:42:18 --- A patch referencing this bug report has been merged in Linux v3.5-rc1:
commit e59717550e5cf0e7159c5b7af1d1ead35fef49dd
Author: Avi Kivity <[email protected]> Date: Mon Apr 9 18:40:03 2012 +0300
KVM: x86 emulator: implement MMX MOVQ (opcodes 0f 6f, 0f 7f)
Signature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.
[Bug 42779] KVM domain hangs after loading initrd with Xenomai kernel
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=42779
Alan <[email protected]> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
CC| |[email protected]
Resolution| |CODE_FIXSignature · 3 lines
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug.