KVM entry failed, hardware error
Hi list,
when trying to install Windows7 in a qemu-kvm 1.0.1 installation on Gentoo on my host running
Linux joequad 3.3.7 #1 SMP PREEMPT Sun Jun 3 12:05:59 CEST 2012 x86_64 Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz GenuineIntel GNU/Linux
with a host CPU of
processor : 0vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2003.000
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yesfpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dts tpr_shadow vnmi flexpriority
bogomips : 5680.90
clflush size : 64cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
the installation first works fine but upon first reboot qemu terminates with:
KVM: entry failed, hardware error 0x80000021
If you're runnning a guest on an Intel machine without unrestricted mode support, the failure can be most likely due to the guest entering an invalid state for Intel VT. For example, the guest maybe running in big real mode which is not supported on less recent Intel processors.
EAX=00000010 EBX=00000080 ECX=00000000 EDX=00000080 ESI=0025da4a EDI=0007da4a EBP=00001f20 ESP=00000200 EIP=0000009b EFL=00000002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0020 00000200 0000ffff 00009300
CS =b000 002b0000 0000ffff 0000f300
SS =0020 00000200 0000ffff 0000f300
DS =0020 00000200 0000ffff 00009300
FS =0020 00000200 0000ffff 00009300
GS =0020 00000200 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT= 002b0000 00000027
IDT= 00000000 000003ff
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000
Code=02 00 00 ea 91 00 00 00 18 00 0f 20 c0 66 83 e0 fe 0f 22 c0 <66> 31 c0 8e d8 8e c0 8e d0 66 bc 00 04 00 00 8e e0 8e e8 ea 00 00 00 20 00 00 00 20 4a da
which disassembles to
0: 02 00 add (%eax),%al 2: 00 ea add %ch,%dl 4: 91 xchg %eax,%ecx 5: 00 00 add %al,(%eax) 7: 00 18 add %bl,(%eax) 9: 00 0f add %cl,(%edi) b: 20 c0 and %al,%al d: 66 83 e0 fe and $0xfffffffe,%ax 11: 0f 22 c0 mov %eax,%cr0 14: 66 31 c0 xor %eax,%eax 17: 8e d8 mov %ax,%ds 19: 8e c0 mov %ax,%es 1b: 8e d0 mov %ax,%ss 1d: 66 bc 00 04 00 00 mov $0x400,%esp 23: 8e e0 mov %ax,%fs 25: 8e e8 mov %ax,%gs 27: ea 00 00 00 20 ljmp $0x2000,$0x0
Since the instruction clears PE of %cr0, I'm pretty sure it's trying to enter real mode again (for a very brief period no doubt) before firing up the OS. Since I believe my CPU is too old and does not support "Unrestricted Guest", this causes qemu to barf.
Is this assessment correct? Is there some way of getting qemu to switch to "soft" emulation in real mode and to use KVM in protected mode? Or is it impossible to run Windows 7 with kvm on a Core Quad (which possibly doesn't support "Unrestricted Guest")?
For reference, there's at least one more person having an identical problem (I've also posted in that thread):
http://forums.gentoo.org/viewtopic-p-7054198.html
To find out if my CPU really does not suport "Unrestricted Guest" I've also posted http://software.intel.com/en-us/forums/showthread.php?t=105687
Any help is greatly appreciated,
Best regards,
Joe
Quoted 4 lines · Johannes Bauer
Hi list,
when trying to install Windows7 in a qemu-kvm 1.0.1 installation on Gentoo on my host running
Is that a 32-bit or 64-bit Windows 7?
Quoted 46 lines
Linux joequad 3.3.7 #1 SMP PREEMPT Sun Jun 3 12:05:59 CEST 2012
x86_64Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz GenuineIntel GNU/Linuxwith a host CPU of
processor : 0
vendor_id: GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz
stepping : 10
microcode :0xa07
cpu MHz : 2003.000
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yesfpu_exception: yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lmconstant_tscarch_perfmonpebs btsrep_goodnopl aperfmperf pni dtes64 monitords_cplvmx smx est tm2 ssse3 cx16 xtpr pdcmsse4_1xsavelahf_lmdtstpr_shadowvnmi flexpriority
bogomips : 5680.90
clflush size : 64cache_alignment: 64
address sizes : 36 bits physical, 48 bits virtual
power management:the installation first works fine but upon first reboot qemu terminates with:
KVM: entry failed, hardware error
0x80000021If you're runnning a guest on an Intel machine without unrestricted mode support, the failure can be most likely due to the guest entering an invalid state for Intel VT. For example, the guest maybe running in big real mode which is not supported on less recent Intel processors.
Quoted 5 lines
EAX=00000010 EBX=00000080 ECX=00000000 EDX=00000080 ESI=0025da4a EDI=0007da4a EBP=00001f20 ESP=00000200 EIP=0000009b EFL=00000002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0020 00000200
0000ffff00009300
CS =b000002b00000000ffff0000f300
Pre-unrestricted mode hosts require cs.selector (first number) * 16 == cs.base (second number). This clearly isn't the case here.
Quoted 7 lines
Since the instruction clears PE of %cr0, I'm pretty sure it's trying to enter real mode again (for a very brief period no doubt) before firing up the OS. Since I believe my CPU is too old and does not support "Unrestricted Guest", this causes qemu to barf.
Is this assessment correct? Is there some way of getting qemu to switch to "soft" emulation in real mode and to use KVM in protected mode?
# rmmod kvm_intel # modprobe kvm_intel emulate_invalid_guest_state=1
Warning: experimental code.
Quoted 3 lines
Or is
it impossible to run Windows 7 with kvm on a Core Quad (which possibly doesn't support "Unrestricted Guest")?
It should be possible. Did you do anything special with the guest (install any software?) before rebooting? Is this repeatable?
Quoted 3 lines
For reference, there's at least one more person having an identical problem (I've also posted in that thread):
http://forums.gentoo.org/viewtopic-p-7054198.html
Same cs.sel/cs.base as your report.
Quoted 2 lines
To find out if my CPU really does not suport "Unrestricted Guest" I've also posted http://software.intel.com/en-us/forums/showthread.php?t=105687
You can run vmxcap [1] and see.
[1]
http://git.qemu.org/?p=qemu.git;a=blob_plain;f=scripts/kvm/vmxcap;hb=HEAD
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 4 lines · Avi Kivity
Quoted 2 lines
when trying to install Windows7 in a qemu-kvm 1.0.1 installation on Gentoo on my host running
Is that a 32-bit or 64-bit Windows 7?
64 bit. But I've also (in despair) tried installing 32 Bit Windows 7, with the exact same effect.
Quoted 8 lines
Quoted 5 lines
EAX=00000010 EBX=00000080 ECX=00000000 EDX=00000080 ESI=0025da4a EDI=0007da4a EBP=00001f20 ESP=00000200 EIP=0000009b EFL=00000002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0020 00000200
0000ffff00009300
CS =b000002b00000000ffff0000f300Pre-unrestricted mode hosts require cs.selector (first number) * 16 == cs.base (second number). This clearly isn't the case here.
I've checked with vmxcap and my PC indeed does not support unrestricted mode. My Laptop (on which the identical image works perfectly) does.
Quoted 4 lines
# rmmod kvm_intel # modprobe kvm_intel emulate_invalid_guest_state=1Warning: experimental code.
Tried it out -- I don't get the error anymore, but qemu just hangs on boot with 100% CPU. :-(
Quoted 6 lines
Quoted 3 lines
Or is
it impossible to run Windows 7 with kvm on a Core Quad (which possibly doesn't support "Unrestricted Guest")?It should be possible. Did you do anything special with the guest (install any software?) before rebooting? Is this repeatable?
Nothing at all, just a standard installation. Did not even get to the point where I *could* install software (it reboots during the installation and that's where it doesn't come up anymore).
I even tried to completing the installation on my laptop (which works), then copy the image back on my desktop PC: Same error.
Any hints on how I could debug the hang with the emulation of invalid guest state? Or is there a more recent version (git?) than a 3.3.7 kernel that I should try out?
Thanks for your help,
Best regards,
Joe
Quoted 18 lines · Johannes Bauer
Quoted 4 lines · Avi Kivity
Quoted 2 lines
when trying to install Windows7 in a qemu-kvm 1.0.1 installation on Gentoo on my host running
Is that a 32-bit or 64-bit Windows 7?
64 bit. But I've also (in despair) tried installing 32 Bit Windows 7, with the exact same effect.
Quoted 8 lines
Quoted 5 lines
EAX=00000010 EBX=00000080 ECX=00000000 EDX=00000080 ESI=0025da4a EDI=0007da4a EBP=00001f20 ESP=00000200 EIP=0000009b EFL=00000002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0020 00000200
0000ffff00009300
CS =b000002b00000000ffff0000f300Pre-unrestricted mode hosts require cs.selector (first number) * 16 == cs.base (second number). This clearly isn't the case here.
Gleb points out that cs.base == gdt.base, which doesn't make a lot of sense. So some corruption has happened earlier. Please follow
http://www.linux-kvm.org/page/Tracing and post the last few thousand lines somewhere. Trace a uniprocessor guest if the problem reproduces that way.
Quoted 11 lines
I've checked with vmxcap and my PC indeed does not support unrestricted mode. My Laptop (on which the identical image works perfectly) does.
Quoted 4 lines
# rmmod kvm_intel # modprobe kvm_intel emulate_invalid_guest_state=1Warning: experimental code.
Tried it out -- I don't get the error anymore, but qemu just hangs on boot with 100% CPU. :-(
Not too surprising, this hasn't been tested in quite a while.
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
Or is
it impossible to run Windows 7 with kvm on a Core Quad (which possibly doesn't support "Unrestricted Guest")?It should be possible. Did you do anything special with the guest (install any software?) before rebooting? Is this repeatable?
Quoted 6 lines
Nothing at all, just a standard installation. Did not even get to the point where I *could* install software (it reboots during the installation and that's where it doesn't come up anymore).
I even tried to completing the installation on my laptop (which works), then copy the image back on my desktop PC: Same error.
That should help the tracing effort.
Quoted 4 lines
Any hints on how I could debug the hang with the emulation of invalid guest state? Or is there a more recent version (git?) than a 3.3.7 kernel that I should try out?
3.3.7 should definitely work. Let's see what the traces tell us.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 17 lines · Avi Kivity
Quoted 16 lines · Johannes Bauer
Quoted 4 lines · Avi Kivity
Quoted 2 lines
when trying to install Windows7 in a qemu-kvm 1.0.1 installation on Gentoo on my host running
Is that a 32-bit or 64-bit Windows 7?
64 bit. But I've also (in despair) tried installing 32 Bit Windows 7, with the exact same effect.
Quoted 6 lines
Quoted 5 lines
EAX=00000010 EBX=00000080 ECX=00000000 EDX=00000080 ESI=0025da4a EDI=0007da4a EBP=00001f20 ESP=00000200 EIP=0000009b EFL=00000002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0020 00000200
0000ffff00009300
CS =b000002b00000000ffff0000f300
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Pre-unrestricted mode hosts require cs.selector (first number) * 16 == cs.base (second number). This clearly isn't the case here.
Gleb points out that cs.base == gdt.base, which doesn't make a lot of sense. So some corruption has happened earlier. Please follow
http://www.linux-kvm.org/page/Tracing and post the last few thousand lines somewhere. Trace a uniprocessor guest if the problem reproduces that way.
Tried that -- and failed miserably.
Something in the trace format appears to have changed. When using trace-cmd report, I first get a parser warning:
trace-cmd: No such file or directory
bad op token {
failed to read event print fmt for kvm_emulate_insn
version = 6
CPU 0 is empty
cpus=4
The ENOENT errno doesn't correspond to that error, strace showed that occuring much earlier in something unrelated. The warning is generated by:
#1 0x0000000000412c66 in process_op (event=0x6537c0, arg=0x6564f0, tok=0x7fffffffd508) at /home/joe/qemu/trace/trace-cmd/parse-events.c:1686 #2 0x0000000000414eef in process_arg_token (event=0x6537c0, arg=0x6564f0, tok=0x7fffffffd598, type=EVENT_OP) at /home/joe/qemu/trace/trace-cmd/parse-events.c:2604
#3 0x0000000000412727 in process_arg (event=0x6537c0, arg=0x6564f0, tok=0x7fffffffd598) at /home/joe/qemu/trace/trace-cmd/parse-events.c:1514 #4 0x00000000004146b4 in process_paren (event=0x6537c0, arg=0x6564f0, tok=0x7fffffffd5f8) at /home/joe/qemu/trace/trace-cmd/parse-events.c:2352 #5 0x0000000000414eb0 in process_arg_token (event=0x6537c0, arg=0x6564f0, tok=0x7fffffffd678, type=EVENT_DELIM) at /home/joe/qemu/trace/trace-cmd/parse-events.c:2596
#6 0x0000000000412727 in process_arg (event=0x6537c0, arg=0x6564f0, tok=0x7fffffffd678) at /home/joe/qemu/trace/trace-cmd/parse-events.c:1514 #7 0x0000000000414f87 in event_read_print_args (event=0x6537c0, list=0x656490) at /home/joe/qemu/trace/trace-cmd/parse-events.c:2637 #8 0x000000000041524f in event_read_print (event=0x6537c0) at /home/joe/qemu/trace/trace-cmd/parse-events.c:2723
#9 0x0000000000419040 in pevent_parse_event (pevent=0x63a0d0,
buf=0x653390 "name: kvm_emulate_insn\nID:
29\nformat:\n\tfield:unsigned short common_type;\toffset:0;\tsize:2;\tsigned:0;\n\tfield:unsigned char common_flags;\toffset:2;\tsize:1;\tsigned:0;\n\tfield:unsigned char common_preempt_"..., size=1053, sys=0x640790 "kvm") at /home/joe/qemu/trace/trace-cmd/parse-events.c:4626
#10 0x000000000042190a in read_event_file (handle=0x63a030, system=0x640790 "kvm", size=1053, print=0) at /home/joe/qemu/trace/trace-cmd/trace-input.c:386
#11 0x0000000000421a90 in read_event_files (handle=0x63a030, print=0) at /home/joe/qemu/trace/trace-cmd/trace-input.c:450
#12 0x0000000000421cb3 in tracecmd_read_headers (handle=0x63a030) at /home/joe/qemu/trace/trace-cmd/trace-input.c:539
#13 0x000000000040c6a5 in trace_report (argc=2, argv=0x7fffffffd9d8) at /home/joe/qemu/trace/trace-cmd/trace-read.c:1025
#14 0x00000000004057e7 in main (argc=2, argv=0x7fffffffd9d8) at /home/joe/qemu/trace/trace-cmd/trace-cmd.c:147
The decoding that follows is complete garbage, which is why I think it's of no use to you (i.e. the insn= part just displays some random undefined memory from the looks).
Therefore, I've uploaded the compressed trace.dat file, so you can maybe have a look why the "report" tool barfs and interpret it correctly. I can't figure it out. The trace is here:
http://spornkuller.de/trace.dat.bz2
The trace was generated with a buffer of 40000 (double of what was given in the example), when it was lower I had overruns. This way I had none. Also, to simulate a single cpu, I've run qemu with "-smp 1", hopefully that's correct.
Quoted 4 lines
Quoted 2 lines
I even tried to completing the installation on my laptop (which works), then copy the image back on my desktop PC: Same error.
That should help the tracing effort.
Should I also generate a trace there and look for the corresponding opcodes that fail on my main machine?
Best regards,
Joe
Quoted 6 lines · Johannes Bauer
Therefore, I've uploaded the compressed trace.dat file, so you can maybe have a look why the "report" tool barfs and interpret it correctly. I can't figure it out. The trace is here:
I can read this trace. Can you do "info pci" in qemu's monitor after failure? What is your command line?
--
Gleb.
Quoted 9 lines · Gleb Natapov
Quoted 6 lines · Johannes Bauer
Therefore, I've uploaded the compressed trace.dat file, so you can maybe have a look why the "report" tool barfs and interpret it correctly. I can't figure it out. The trace is here:
I can read this trace. Can you do "info pci" in qemu's monitor after failure? What is your command line?
Also after the failure:
x/256b 0x2b0000
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 8 lines · Gleb Natapov
Quoted 6 lines · Johannes Bauer
Therefore, I've uploaded the compressed trace.dat file, so you can maybe have a look why the "report" tool barfs and interpret it correctly. I can't figure it out. The trace is here:
I can read this trace.
Hm, weird. But good that it works on your side. I get a lot of:
trace-cmd: No such file or directory
bad op token {
failed to read event print fmt for kvm_emulate_insn
version = 6
CPU 0 is empty
cpus=4
qemu-system-x86-10775 [001] 2512.220779: kvm_fpu: load qemu-system-x86-10775 [001] 2512.220782: kvm_entry: vcpu 0 qemu-system-x86-10775 [001] 2512.220785: kvm_exit: reason EXCEPTION_NMI rip 0xfff0 info ffff0 80000b0e qemu-system-x86-10775 [001] 2512.220787: kvm_page_fault: address ffff0 error_code 14 qemu-system-x86-10775 [001] 2512.220796: kvm_entry: vcpu 0 qemu-system-x86-10775 [001] 2512.220798: kvm_exit: reason EXCEPTION_NMI rip 0xc81e info 0 80000b0d qemu-system-x86-10775 [001] 2512.220803: kvm_emulate_insn: [FAILED TO PARSE] rip=51230 csbase=983040 len=3 insn= �f%����"���f flags=0 failed=0 qemu-system-x86-10775 [001] 2512.220806: kvm_entry: vcpu 0 qemu-system-x86-10775 [001] 2512.220807: kvm_exit: reason
EXCEPTION_NMI rip 0xc827 info 0 80000b0d
qemu-system-x86-10775 [001] 2512.220808: kvm_emulate_insn: [FAILED
TO PARSE] rip=51239 csbase=983040 len=3 insn="���f�flags=0 failed=0 [...]
Quoted 2 lines
Can you do "info pci" in qemu's monitor
after failure?
(qemu) info pci
Bus 0, device 0, function 0:
Host bridge: PCI device 8086:1237
id ""
Bus 0, device 1, function 0:
ISA bridge: PCI device 8086:7000
id ""
Bus 0, device 1, function 1:
IDE controller: PCI device 8086:7010
BAR4: I/O at 0xc000 [0xc00f].
id ""
Bus 0, device 1, function 3:
Bridge: PCI device 8086:7113
IRQ 9.
id ""
Bus 0, device 2, function 0:
VGA controller: PCI device 1013:00b8
BAR0: 32 bit prefetchable memory at 0xf0000000 [0xf1ffffff].
BAR1: 32 bit memory at 0xf2000000 [0xf2000fff].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
id ""
Bus 0, device 3, function 0:
Ethernet controller: PCI device 8086:100e
IRQ 11.
BAR0: 32 bit memory at 0xf2020000 [0xf203ffff].
BAR1: I/O at 0xc040 [0xc07f].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0001fffe].
id ""
Bus 0, device 4, function 0:
SCSI controller: PCI device 1af4:1001
IRQ 11.
BAR0: I/O at 0xc080 [0xc0bf].
BAR1: 32 bit memory at 0xf2060000 [0xf2060fff].
id ""Quoted 1 line
What is your command line?
bin/qemu-system-x86_64 -cpu host -enable-kvm -net nic -net user,smb=Share,restrict=on -drive media=disk,file=Windows7_x32.qcow2,if=virtio -m 2048 -smp 1 -nographic
(added "-nographic" to be able to enter the console)
Also, as per Avi's request:
(qemu) x/256b 0x2b000000000000002b0000: 0xeb 0x26 0x27 0x00 0x00 0x00 0x2b 0x00 00000000002b0008: 0xff 0xff 0x00 0x00 0x00 0x9a 0xcf 0x00 00000000002b0010: 0xff 0xff 0x00 0x00 0x00 0x92 0xcf 0x00 00000000002b0018: 0xff 0xff 0x00 0x00 0x2b 0x9f 0x00 0x00 00000000002b0020: 0xff 0xff 0x00 0x02 0x00 0x93 0x00 0x00 00000000002b0028: 0x8a 0x15 0x68 0xbc 0x00 0x00 0xa1 0xbf 00000000002b0030: 0x00 0x2b 0x00 0x85 0xc0 0x74 0x06 0x8b 00000000002b0038: 0x1d 0xbb 0x00 0x2b 0x00 0xa1 0xc7 0x00 00000000002b0040: 0x2b 0x00 0x85 0xc0 0x74 0x06 0x8b 0x15 00000000002b0048: 0xc3 0x00 0x2b 0x00 0xbe 0x00 0x00 0x20 00000000002b0050: 0x00 0x31 0xc0 0x31 0xff 0x66 0x8b 0x3d 00000000002b0058: 0xb5 0x00 0x2b 0x00 0xc1 0xe7 0x04 0x66 00000000002b0060: 0xa1 0xb3 0x00 0x2b 0x00 0x01 0xc7 0x8b 00000000002b0068: 0x0d 0xb7 0x00 0x2b 0x00 0xfc 0xf3 0xa4 00000000002b0070: 0x0f 0x01 0x15 0x02 0x00 0x2b 0x00 0x66 00000000002b0078: 0xb8 0x20 0x00 0x8e 0xd8 0x8e 0xc0 0x8e 00000000002b0080: 0xe0 0x8e 0xe8 0x8e 0xd0 0xbc 0x00 0x02 00000000002b0088: 0x00 0x00 0xea 0x91 0x00 0x00 0x00 0x18 00000000002b0090: 0x00 0x0f 0x20 0xc0 0x66 0x83 0xe0 0xfe 00000000002b0098: 0x0f 0x22 0xc0 0x66 0x31 0xc0 0x8e 0xd8 00000000002b00a0: 0x8e 0xc0 0x8e 0xd0 0x66 0xbc 0x00 0x04 00000000002b00a8: 0x00 0x00 0x8e 0xe0 0x8e 0xe8 0xea 0x00 00000000002b00b0: 0x00 0x00 0x20 0x00 0x00 0x00 0x20 0x4a 00000000002b00b8: 0xda 0x05 0x00 0x80 0x00 0x00 0x00 0x01 00000000002b00c0: 0x00 0x00 0x00 0x80 0x00 0x00 0x00 0x01 00000000002b00c8: 0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x90 00000000002b00d0: 0x90 0x90 0x55 0x89 0xe5 0x57 0x8b 0x7d 00000000002b00d8: 0x0c 0x56 0x53 0x8b 0x5d 0x08 0x6a 0x23 00000000002b00e0: 0x6a 0x00 0x68 0x80 0x05 0x00 0x00 0xe8 00000000002b00e8: 0xf3 0xcf 0x00 0x00 0x83 0xc4 0x0c 0x83 00000000002b00f0: 0x3d 0xe0 0x04 0x03 0x00 0x00 0xc6 0x05 00000000002b00f8: 0x80 0x05 0x00 0x00 0x13 0x74 0x0d 0x53
Best regards,
Joe
Quoted 5 lines · Johannes Bauer
Quoted 1 line
What is your command line?
bin/qemu-system-x86_64 -cpu host -enable-kvm -net nic -net user,smb=Share,restrict=on -drive media=disk,file=Windows7_x32.qcow2,if=virtio -m 2048 -smp 1 -nographic
Just noticed that the output I just provided was for the 32 Bit version of Windows 7. Did the same (info pci and the memdump) for the 64 Bit version again and diffed them with meld -- they're *perfectly* identical. Just so there isn't any confusion.
Best regards,
Joe
Quoted 8 lines · Johannes Bauer
Quoted 7 lines · Gleb Natapov
Quoted 6 lines · Johannes Bauer
Therefore, I've uploaded the compressed trace.dat file, so you can maybe have a look why the "report" tool barfs and interpret it correctly. I can't figure it out. The trace is here:
Quoted 9 lines
Quoted 1 line
I can read this trace.
Hm, weird. But good that it works on your side. I get a lot of:
trace-cmd: No such file or directory
bad op token {
failed to read event print fmt forkvm_emulate_insn
version = 6
CPU 0 is empty
You need to install the plugins for trace-cmd:
make install_plugins
you could also try setting:
export TRACE_CMD_PLUGIN_DIR=/path/to/plugins
David
Quoted 5 lines · David Ahern
You need to install the plugins for trace-cmd:
makeinstall_pluginsyou could also try setting:
export TRACE_CMD_PLUGIN_DIR=/path/to/plugins
Aha, with that I get more messages:
trace-cmd: No such file or directory
cound not load plugin '/home/joe/qEmu/trace/trace-cmd/plugin_kvm.so'/home/joe/qemu/trace/trace-cmd/plugin_kvm.so: undefined symbol:ud_translate_att
However, during the build process, udis86 is linked in like it should be:
[...]
gcc -g -Wall -I. '-DPLUGIN_DIR="/usr/local/share/trace-cmd/plugins"' -DHAVE_UDIS86 -ludis86 -DHAVE_BLK_TC_FLUSH -shared -nostartfiles -o plugin_blk.so plugin_blk.o
But when looking, I just noticed that it does not appear in the imports
$ ldd plugin_kvm.so linux-vdso.so.1 => (0x00007fff3cfff000) libc.so.6 => /lib64/libc.so.6 (0x00007f6429e1c000) /lib64/ld-linux-x86-64.so.2 (0x00007f642a3e4000)
Nor do I actually *have* the .so of udis86 in my /usr/lib directory. Gentoo by default just built the static library. Weird! I'll recompile myself and then it'll work for sure.
Best regards,
Joe
Quoted 7 lines · Johannes Bauer
Quoted 4 lines
# rmmod kvm_intel # modprobe kvm_intel emulate_invalid_guest_state=1Warning: experimental code.
Tried it out -- I don't get the error anymore, but qemu just hangs on boot with 100% CPU. :-(
Please retry with
git://git.kernel.org/pub/scm/virt/kvm/kvm.git big-real-mode
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 11 lines · Avi Kivity
Quoted 7 lines
Quoted 4 lines
# rmmod kvm_intel # modprobe kvm_intel emulate_invalid_guest_state=1Warning: experimental code.
Tried it out -- I don't get the error anymore, but qemu just hangs on boot with 100% CPU. :-(
Please retry with
git://git.kernel.org/pub/scm/virt/kvm/kvm.gitbig-real-mode
Same thing:
$ uname -a Linux joequad 3.5.0-rc1-46053-gf2ebd42 #1 SMP PREEMPT Wed Jun 6 19:49:21 CEST 2012 x86_64 Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz GenuineIntel GNU/Linux
$ cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.5.0-rc1-46053-gf2ebd42 root=/dev/sda1 ro kvm-intel.emulate_invalid_guest_state=1
QEmu hangs with 100% CPU. Is there anything I can try to be able to tell you *where* it hangs?
Best regards,
Johannes
Quoted 21 lines · Johannes Bauer
Quoted 11 lines · Avi Kivity
Quoted 7 lines
Quoted 4 lines
# rmmod kvm_intel # modprobe kvm_intel emulate_invalid_guest_state=1Warning: experimental code.
Tried it out -- I don't get the error anymore, but qemu just hangs on boot with 100% CPU. :-(
Please retry with
git://git.kernel.org/pub/scm/virt/kvm/kvm.gitbig-real-modeSame thing:
$ uname -a Linux joequad 3.5.0-rc1-46053-gf2ebd42 #1 SMP PREEMPT Wed Jun 6 19:49:21 CEST 2012 x86_64 Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz GenuineIntel GNU/Linux
Quoted 7 lines
$ cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-3.5.0-rc1-46053-gf2ebd42 root=/dev/sda1 ro kvm-intel.emulate_invalid_guest_state=1QEmu hangs with 100% CPU. Is there anything I can try to be able to tell you *where* it hangs?
add -monitor stdio to the command line and then:
(qemu) info registers
(qemu) x/20i 0xcsbase + $eip
(for csbase substitute the second value for CS in 'info registers')
Run info registers a few times and note whether eip changes or not.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 6 lines · Avi Kivity
add -monitor stdio to the command line and then:
(qemu) info registers
(qemu) x/20i 0xcsbase + $eipRun info registers a few times and note whether eip changes or not.
It does not. Here's where it hangs:
(qemu) info registers
EAX=000023de EBX=00000b70 ECX=00000b90 EDX=00000002 ESI=002523de EDI=00000b84 EBP=0000146e ESP=0000146e EIP=000008d7 EFL=00000202 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =23de 00023de0 0000ffff 0000f300
CS =2000 00020000 0000ffff 0000f300
SS =23de 00023de0 0000ffff 0000f300
DS =23de 00023de0 0000ffff 0000f300
FS =0060 00023de0 0000ffff 00009300
GS =0060 00023de0 0000ffff 00009300
LDT=0000 00000000 ffffffff 00c00000
TR =0040 feffd000 00002088 00008b00
GDT= 0001f000 0000007f
IDT= 00000000 0000ffff
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80 FPR0=0000000000000000 0000 FPR1=0000000000000000 0000 FPR2=0000000000000000 0000 FPR3=0000000000000000 0000 FPR4=0000000000000000 0000 FPR5=0000000000000000 0000 FPR6=0000000000000000 0000 FPR7=0000000000000000 0000 XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000 XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000 XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000 XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
(qemu) x/20i 0x20000 + $eip
0x00000000000208d7: leave 0x00000000000208d8: ret 0x00000000000208d9: enter $0x0,$0x0 0x00000000000208dd: push %ebp 0x00000000000208df: push %ebx 0x00000000000208e1: push %esi 0x00000000000208e3: push %edi 0x00000000000208e5: mov %esp,%ebx 0x00000000000208e8: mov %ebx,%edi 0x00000000000208eb: add $0x14,%edi 0x00000000000208ef: addr32 mov (%edi),%eax 0x00000000000208f3: mov $0x1480,%sp 0x00000000000208f6: xor %bp,%bp 0x00000000000208f8: movzwl %bp,%ebp 0x00000000000208fc: movzwl %sp,%esp 0x0000000000020900: push %ebx 0x0000000000020902: push %eax 0x0000000000020904: call 0x20919 0x0000000000020907: add $0x4,%sp 0x000000000002090a: pop %ebx
And this is where it came from and tries to return to:
(qemu) x /8hx 0x23de0 + $esp000000000002524e: 0x1474 0x092a 0x0001 0x0000 0x0907 0x4970 0x0002 0x0b70
(qemu) x/20i 0x20000 + 0x92a - 0x15
0x0000000000020915: pop %ebp 0x0000000000020917: leave 0x0000000000020918: ret 0x0000000000020919: enter $0x0,$0x0 0x000000000002091d: mov 0x1510,%ax 0x0000000000020920: push %ax 0x0000000000020921: and %ax,%ax 0x0000000000020923: je 0x2092a 0x0000000000020927: call 0x20871 0x000000000002092a: push %bx 0x000000000002092b: push %di 0x000000000002092c: push %si 0x000000000002092d: push %ds 0x000000000002092e: push %es 0x000000000002092f: push %bp 0x0000000000020930: mov 0x4(%bp),%eax 0x0000000000020934: mov %ax,%bp 0x0000000000020936: and $0xf,%bp 0x0000000000020939: shr $0x4,%eax 0x000000000002093d: mov %ax,%ds
Here's the whole function that causes the hangup:
(qemu) x/39i 0x20000 + 0x871
0x0000000000020871: enter $0x0,$0x0 0x0000000000020875: push %ebx 0x0000000000020877: mov 0x1510,%ax 0x000000000002087a: and %ax,%ax 0x000000000002087c: je 0x208d5 0x0000000000020880: sgdtw 0x1500 0x0000000000020885: sidtw 0x1508 0x000000000002088a: movw $0x0,0x1510 0x0000000000020890: mov %cr0,%eax 0x0000000000020893: mov %eax,0x1514 0x0000000000020897: and $0x7ffffffe,%eax 0x000000000002089d: mov %eax,%cr0 0x00000000000208a0: jmp 0x208a5 0x00000000000208a2: nop 0x00000000000208a3: nop 0x00000000000208a4: nop 0x00000000000208a5: mov %cr3,%eax 0x00000000000208a8: nop 0x00000000000208a9: nop 0x00000000000208aa: nop 0x00000000000208ab: nop 0x00000000000208ac: mov %eax,%cr3 0x00000000000208af: pushw 0x1536 0x00000000000208b3: pop %es 0x00000000000208b4: mov $0x8c6,%bx 0x00000000000208b7: mov 0x1536,%ax 0x00000000000208ba: mov %ax,%es:-0x2(%bx) 0x00000000000208be: ljmp *%es:-0x4(%bx) 0x00000000000208c2: (bad) 0x00000000000208c3: or %al,(%bx,%si) 0x00000000000208c5: and %ah,0x1534(%bx,%di) 0x00000000000208c9: mov %ax,%ds 0x00000000000208cb: mov %ax,%ss 0x00000000000208cd: mov %ax,%es 0x00000000000208cf: lidtw 0x14f8 0x00000000000208d4: sti 0x00000000000208d5: pop %ebx 0x00000000000208d7: leave 0x00000000000208d8: ret
Best regards,
Joe
Quoted 7 lines · Johannes Bauer
Quoted 5 lines · Avi Kivity
add -monitor stdio to the command line and then:
(qemu) info registers
(qemu) x/20i 0xcsbase + $eip
Quoted 6 lines
Quoted 1 line
Run info registers a few times and note whether eip changes or not.
It does not. Here's where it hangs:
(qemu) x/20i
0x20000+ $eip0x00000000000208d7: leave
Pretty straightforward, we need to emulate the leave instruction. I'll update the branch and notify you.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 10 lines · Avi Kivity
Quoted 9 lines · Johannes Bauer
Quoted 6 lines · Avi Kivity
add -monitor stdio to the command line and then:
(qemu) info registers
(qemu) x/20i 0xcsbase + $eipRun info registers a few times and note whether eip changes or not.
Quoted 8 lines
Quoted 4 lines
It does not. Here's where it hangs:
(qemu) x/20i
0x20000+ $eip0x00000000000208d7: leavePretty straightforward, we need to emulate the leave instruction. I'll update the branch and notify you.
Please try the big-real-mode branch again. It contains emulation for the missing instruction, plus a bunch of tweaks which allowed it to boot Fedora 17 smp with emulate_invalid_guest_state=1.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 3 lines · Avi Kivity
Please try the big-real-mode branch again. It contains emulation for the missing instruction, plus a bunch of tweaks which allowed it to boot Fedora 17 smp with emulate_invalid_guest_state=1.
Progress!
So now I'm on
Linux joequad 3.5.0-rc1-46078-g54d5c7c #2 SMP PREEMPT Thu Jun 7 17:11:38 CEST 2012 x86_64 Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz GenuineIntel GNU/Linux
And see the following (kind of weird) behavior: I can boot up now. But it's bound to some condition:
When not using "-monitor stdio", I have the same behavior as before.
When using "-monitor stdio", it locks up occasionaly, but appears to resume operation whenever I type "info registers". That way I was able to boot up (by entering about 50 times "info registers", then it had left real mode and booted up normally).
I was first a bit confused, but this is definitely what I'm seeing:
Because of shutting down the VM I had put the guest in a "Windows Recovery" mode where a text-mode progress bar appears with the text "Windows is loading files...". The bar locks up -- when I type "info registers", it progresses about 2-4 characters and locks up again (until it's fully loaded, then it just works).
Note that "c" does NOT cause the VM to resume, only "info registers" does. dmesg shows nothing out of the ordinary.
But I think we're getting somewhere! Thank you very much for your support already!
Best regards,
Joe
Quoted 7 lines · Johannes Bauer
Quoted 3 lines · Avi Kivity
Please try the big-real-mode branch again. It contains emulation for the missing instruction, plus a bunch of tweaks which allowed it to boot Fedora 17 smp with emulate_invalid_guest_state=1.
Progress!
Great!
Quoted 23 lines
So now I'm on
Linux joequad 3.5.0-rc1-46078-g54d5c7c #2 SMP PREEMPT Thu Jun 7 17:11:38 CEST 2012
x86_64Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz GenuineIntel GNU/LinuxAnd see the following (kind of weird) behavior: I can boot up now. But it's bound to some condition:
When not using "-monitor stdio", I have the same behavior as before.
When using "-monitor stdio", it locks up occasionaly, but appears to resume operation whenever I type "info registers". That way I was able to boot up (by entering about 50 times "info registers", then it had left real mode and booted up normally).
I was first a bit confused, but this is definitely what I'm seeing:
Because of shutting down the VM I had put the guest in a "Windows Recovery" mode where a text-mode progress bar appears with the text "Windows is loading files...". The bar locks up -- when I type "info registers", it progresses about 2-4 characters and locks up again (until it's fully loaded, then it just works).
Quoted 2 lines
Note that "c" does NOT cause the VM to resume, only "info registers" does. dmesg shows nothing out of the ordinary.
I'm guessing this is 5152902652. Try bumping 'unsigned count = 130' (by adding zeros at the end, don't bother with anything less). If you increase it too much qemu may hang; but kill -9 should unfreeze it.
Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.
Quoted 6 lines · Avi Kivity
Quoted 2 lines
Note that "c" does NOT cause the VM to resume, only "info registers" does. dmesg shows nothing out of the ordinary.
I'm guessing this is 5152902652. Try bumping 'unsigned count = 130' (by adding zeros at the end, don't bother with anything less). If you increase it too much qemu may hang; but kill -9 should unfreeze it.
Doesn't seem to be right -- still got the same problem. I first bumped it up to 1300 and inserted debugging output to see how many cycles are actually spent in the loop. It enters the emulation mode so frequently (and leaves it again) that the dmesg buffer ran over (128kB). So I changed the debugging to give me the lowest cycle count that it ever has after the loop:
handle_invalid_guest_state: emulation left, new low count 1295 handle_invalid_guest_state: emulation left, new low count 1292 handle_invalid_guest_state: emulation left, new low count 1291 handle_invalid_guest_state: emulation left, new low count 1245
Which means that it spends a maximum of 55 cycles in the loop (well below the original 130 even). So my change had no effect. Any other ideas maybe?
Best regards,
Joe
Quoted 16 lines · Johannes Bauer
Quoted 6 lines · Avi Kivity
Quoted 2 lines
Note that "c" does NOT cause the VM to resume, only "info registers" does. dmesg shows nothing out of the ordinary.
I'm guessing this is 5152902652. Try bumping 'unsigned count = 130' (by adding zeros at the end, don't bother with anything less). If you increase it too much qemu may hang; but kill -9 should unfreeze it.
Doesn't seem to be right -- still got the same problem. I first bumped it up to 1300 and inserted debugging output to see how many cycles are actually spent in the loop. It enters the emulation mode so frequently (and leaves it again) that the dmesg buffer ran over (128kB). So I changed the debugging to give me the lowest cycle count that it ever has after the loop:
Quoted 9 lines
handle_invalid_guest_state: emulation left, new low count 1295handle_invalid_guest_state: emulation left, new low count 1292handle_invalid_guest_state: emulation left, new low count 1291handle_invalid_guest_state: emulation left, new low count 1245Which means that it spends a maximum of 55 cycles in the loop (well below the original 130 even). So my change had no effect. Any other ideas maybe?
Looks like we weren't dealing with interrupts correctly. I pushed some patches, please pull again and retry.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 2 lines · Avi Kivity
Looks like we weren't dealing with interrupts correctly. I pushed some patches, please pull again and retry.
Updated to cf3d9372065470403e0780599ca612553211a10b and it works perfectly for me!
Thank you very much for the support! Very cool!
Best regards,
Joe
Quoted 7 lines · Johannes Bauer
Quoted 2 lines · Avi Kivity
Looks like we weren't dealing with interrupts correctly. I pushed some patches, please pull again and retry.
Updated to
cf3d9372065470403e0780599ca612553211a10band it works perfectly for me!
Good to hear. We'll try to get emulate_invalid_guest_state=1 to be the default for 3.6.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 6 lines · Avi Kivity
Quoted 2 lines
Note that "c" does NOT cause the VM to resume, only "info registers" does. dmesg shows nothing out of the ordinary.
I'm guessing this is 5152902652. Try bumping 'unsigned count = 130' (by adding zeros at the end, don't bother with anything less). If you increase it too much qemu may hang; but kill -9 should unfreeze it.
Okay, here's more clues. They're not exactly meaningful for me, but probably of interest to you.
Besides the "low count value after loop exit" value, I've introduced two counters which both are incremented inside the loop. Also I've modified the loop to be one return only (i.e. ret = foobar; goto out instead of return).
Then when I have a script which continuously echos "info registers" and pipe that output together with qemu, I get this startup:
2012-06-07 21:39:34 handle_invalid_guest_state: emulation left, new loop low count 1295, total of 5 emulated insns 2012-06-07 21:39:34 handle_invalid_guest_state: emulation left, new loop low count 1292, total of 13 emulated insns 2012-06-07 21:39:34 handle_invalid_guest_state: emulation left, 1293 iterations left, 100005 emulated insn, loop low count 1292 2012-06-07 21:39:34 handle_invalid_guest_state: emulation left, new loop low count 1291, total of 131214 emulated insns 2012-06-07 21:39:34 handle_invalid_guest_state: emulation left, new loop low count 1245, total of 131269 emulated insns 2012-06-07 21:39:35 handle_invalid_guest_state: emulation left, 1299 iterations left, 200005 emulated insn, loop low count 1245 2012-06-07 21:39:35 handle_invalid_guest_state: emulation left, 1245 iterations left, 300012 emulated insn, loop low count 1245 2012-06-07 21:39:35 handle_invalid_guest_state: emulation left, 1291 iterations left, 400013 emulated insn, loop low count 1245 2012-06-07 21:39:35 handle_invalid_guest_state: emulation left, 1245 iterations left, 500050 emulated insn, loop low count 1245 2012-06-07 21:39:35 handle_invalid_guest_state: emulation left, 1245 iterations left, 600059 emulated insn, loop low count 1245 2012-06-07 21:39:35 handle_invalid_guest_state: emulation left, 1299 iterations left, 700059 emulated insn, loop low count 1245 2012-06-07 21:39:35 handle_invalid_guest_state: emulation left, 1299 iterations left, 800059 emulated insn, loop low count 1245 2012-06-07 21:39:36 handle_invalid_guest_state: emulation left, 1299 iterations left, 900059 emulated insn, loop low count 1245 2012-06-07 21:39:36 handle_invalid_guest_state: emulation left, 1299 iterations left, 1000059 emulated insn, loop low count 1245 2012-06-07 21:39:36 handle_invalid_guest_state: emulation left, new loop low count 1228, total of 1030349 emulated insns 2012-06-07 21:39:36 handle_invalid_guest_state: emulation left, 1291 iterations left, 1100063 emulated insn, loop low count 1228 2012-06-07 21:39:36 handle_invalid_guest_state: emulation left, 1299 iterations left, 1200063 emulated insn, loop low count 1228 2012-06-07 21:39:36 handle_invalid_guest_state: emulation left, 1299 iterations left, 1300063 emulated insn, loop low count 1228 2012-06-07 21:39:36 handle_invalid_guest_state: emulation left, 1245 iterations left, 1400113 emulated insn, loop low count 1228 2012-06-07 21:39:37 handle_invalid_guest_state: emulation left, 1245
iterations left, 1500145 emulated insn, loop low count 1228
After which it has booted up and does not emulate anymore. Note the maximum time spent in the loop is 72 iterations.
But when I just start up and do not issue ANY "info registers", I get the following output:
2012-06-07 21:41:44 handle_invalid_guest_state: emulation left, new loop low count 1295, total of 5 emulated insns 2012-06-07 21:41:44 handle_invalid_guest_state: emulation left, new loop low count 1292, total of 13 emulated insns 2012-06-07 21:41:44 handle_invalid_guest_state: emulation left, 1293 iterations left, 100004 emulated insn, loop low count 1292 2012-06-07 21:41:44 handle_invalid_guest_state: emulation left, new loop low count 1291, total of 131955 emulated insns 2012-06-07 21:41:44 handle_invalid_guest_state: emulation left, new loop low count 1245, total of 132010 emulated insns 2012-06-07 21:41:44 handle_invalid_guest_state: emulation left, 1299 iterations left, 200004 emulated insn, loop low count 1245 2012-06-07 21:41:45 handle_invalid_guest_state: emulation left, 1299 iterations left, 300004 emulated insn, loop low count 1245 2012-06-07 21:41:45 handle_invalid_guest_state: emulation left, 1299 iterations left, 400004 emulated insn, loop low count 1245 2012-06-07 21:41:45 handle_invalid_guest_state: emulation left, 1299 iterations left, 500004 emulated insn, loop low count 1245 2012-06-07 21:41:45 handle_invalid_guest_state: emulation left, 1299 iterations left, 600004 emulated insn, loop low count 1245 2012-06-07 21:41:45 handle_invalid_guest_state: emulation left, 1299 iterations left, 700004 emulated insn, loop low count 1245 2012-06-07 21:41:45 handle_invalid_guest_state: emulation left, 1299 iterations left, 800004 emulated insn, loop low count 1245 2012-06-07 21:41:45 handle_invalid_guest_state: emulation left, 1299 iterations left, 900004 emulated insn, loop low count 1245 2012-06-07 21:41:45 handle_invalid_guest_state: emulation left, 1299 iterations left, 1000004 emulated insn, loop low count 1245 [...] 2012-06-07 21:42:12 handle_invalid_guest_state: emulation left, 1299 iterations left, 22800004 emulated insn, loop low count 1245 2012-06-07 21:42:12 handle_invalid_guest_state: emulation left, 1299 iterations left, 22900004 emulated insn, loop low count 1245 2012-06-07 21:42:12 handle_invalid_guest_state: emulation left, 1299
iterations left, 23000004 emulated insn, loop low count 1245
In an infinite loop. Which looks to be as if it is continuously exiting after just one iteration (count at leave is 1299). Maybe I'll fiddle some more and am able to provide some insight (probably you already know what's going on, but it won't hurt I guess).
Best regards,
Joe
Quoted 4 lines · Johannes Bauer
In an infinite loop. Which looks to be as if it is continuously exiting after just one iteration (count at leave is 1299). Maybe I'll fiddle some more and am able to provide some insight (probably you already know what's going on, but it won't hurt I guess).
Followup on that: I've shorted the debug output (so it doesn't get wrapped in mail and can be more easily read) and I've enumerated all five possible exists from the loop:
1. intr_window_requested && (kvm_get_rflags(&vmx->vcpu) & X86_EFLAGS_IF)
2. err == EMULATE_DO_MMIO
3. err != EMULATE_DONE
4. signal_pending(current)
5. while loop exit
And added that to the loop. When it hangs, it always exists by #1:
higs: new low cnt 1295, 5 emu insns
higs: new low cnt 1292, 13 emu insns
higs: left count=1293, 100002 emu insns, llc=1292, rsn=5
higs: new low cnt 1291, 135348 emu insns
higs: new low cnt 1245, 135403 emu insns
higs: new low cnt 1228, 183120 emu insns
higs: left count=1291, 200007 emu insns, llc=1228, rsn=5
higs: left count=1299, 300007 emu insns, llc=1228, rsn=1
higs: left count=1299, 400007 emu insns, llc=1228, rsn=1
higs: left count=1299, 500007 emu insns, llc=1228, rsn=1
higs: left count=1299, 600007 emu insns, llc=1228, rsn=1
higs: left count=1299, 700007 emu insns, llc=1228, rsn=1
higs: left count=1299, 800007 emu insns, llc=1228, rsn=1
higs: left count=1299, 900007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1000007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1100007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1200007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1300007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1400007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1500007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1600007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1700007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1800007 emu insns, llc=1228, rsn=1
higs: left count=1299, 1900007 emu insns, llc=1228, rsn=1
higs: left count=1299, 2000007 emu insns, llc=1228, rsn=1
higs: left count=1299, 2100007 emu insns, llc=1228, rsn=1 [...]
If there's any more output I can provide to help you track down the problem at hand, please let me know.
Best regards,
Joe
Quoted 1 line · Avi Kivity
You can run vmxcap [1] and see.
Ah, I forgot to attach the output of vmxcap, which might be of interest. First the host where it does not work:
pin-based controls
External interrupt exiting yes NMI exiting yes Virtual NMIs yes Activate VMX-preemption timer no primary processor-based controls Interrupt window exiting yes Use TSC offsetting yes HLT exiting yes INVLPG exiting yes MWAIT exiting yes RDPMC exiting yes RDTSC exiting yes CR3-load exiting forced CR3-store exiting forced CR8-load exiting yes CR8-store exiting yes Use TPR shadow yes NMI-window exiting yes MOV-DR exiting yes Unconditional I/O exiting yes Use I/O bitmaps yes Monitor trap flag no Use MSR bitmaps yes MONITOR exiting yes PAUSE exiting yes Activate secondary control yes secondary processor-based controls Virtualize APIC accesses yes Enable EPT no Descriptor-table exiting no Virtualize x2APIC mode no Enable VPID no WBINVD exiting yes Unrestricted guest no PAUSE-loop exiting no VM-Exit controls Save debug controls forced Host address-space size yes Load IA32_PERF_GLOBAL_CTRL yes Acknowledge interrupt on exit yes Save IA32_PAT no Load IA32_PAT no Save IA32_EFER no Load IA32_EFER no Save VMX-preemption timer value no VM-Entry controls Load debug controls forced IA-64 mode guest yes Entry to SMM yes Deactivate dual-monitor treatment yes Load IA32_PERF_GLOBAL_CTRL yes Load IA32_PAT no Load IA32_EFER no Miscellaneous data VMX-preemption timer scale (log2) 0 Store EFER.LMA into IA-32e mode guest control no HLT activity state yes Shutdown activity state yes Wait-for-SIPI activity state yes Number of CR3-target values 4 MSR-load/store count recommenation 0 MSEG revision identifier 0 VPID and EPT capabilities Execute-only EPT translations no Page-walk length 4 no Paging-structure memory type UC no Paging-structure memory type WB no 2MB EPT pages no 1GB EPT pages no INVEPT supported no Single-context INVEPT no All-context INVEPT no INVVPID supported no Individual-address INVVPID no Single-context INVVPID no All-context INVVPID no
Single-context-retaining-globals INVVPID no
Then the one where it does:
pin-based controls
External interrupt exiting yes NMI exiting yes Virtual NMIs yes Activate VMX-preemption timer yes primary processor-based controls Interrupt window exiting yes Use TSC offsetting yes HLT exiting yes INVLPG exiting yes MWAIT exiting yes RDPMC exiting yes RDTSC exiting yes CR3-load exiting default CR3-store exiting default CR8-load exiting yes CR8-store exiting yes Use TPR shadow yes NMI-window exiting yes MOV-DR exiting yes Unconditional I/O exiting yes Use I/O bitmaps yes Monitor trap flag yes Use MSR bitmaps yes MONITOR exiting yes PAUSE exiting yes Activate secondary control yes secondary processor-based controls Virtualize APIC accesses yes Enable EPT yes Descriptor-table exiting yes Virtualize x2APIC mode yes Enable VPID yes WBINVD exiting yes Unrestricted guest yes PAUSE-loop exiting no VM-Exit controls Save debug controls default Host address-space size yes Load IA32_PERF_GLOBAL_CTRL yes Acknowledge interrupt on exit yes Save IA32_PAT yes Load IA32_PAT yes Save IA32_EFER yes Load IA32_EFER yes Save VMX-preemption timer value yes VM-Entry controls Load debug controls forced IA-64 mode guest yes Entry to SMM yes Deactivate dual-monitor treatment yes Load IA32_PERF_GLOBAL_CTRL yes Load IA32_PAT yes Load IA32_EFER yes Miscellaneous data VMX-preemption timer scale (log2) 5 Store EFER.LMA into IA-32e mode guest control yes HLT activity state yes Shutdown activity state yes Wait-for-SIPI activity state yes Number of CR3-target values 4 MSR-load/store count recommenation 0 MSEG revision identifier 0 VPID and EPT capabilities Execute-only EPT translations yes Page-walk length 4 yes Paging-structure memory type UC yes Paging-structure memory type WB yes 2MB EPT pages yes 1GB EPT pages no INVEPT supported yes Single-context INVEPT yes All-context INVEPT yes INVVPID supported yes Individual-address INVVPID yes Single-context INVVPID yes All-context INVVPID yes
Single-context-retaining-globals INVVPID yes
Best regards,
Joe