w2k8 - reboots unexpected

Hi

We are running kvm-84 on debian x64 Linux.

We have serveral guest running on this host.

One guest witch is running windows 2008 server and used as a terminal server, reboots sometimes unexpected.

All what I found so far is the following error in the kernel-logfile:

Jun 18 10:33:17 sov07l kernel: [1470365.925930] kvm_handle_exit: Breaking 
out of NMI-blocked state on VCPU 1 after 1 s timeout
Jun 18 10:33:17 sov07l kernel: [1470365.929077] kvm_handle_exit: Breaking 
out of NMI-blocked state on VCPU 2 after 1 s timeout
Jun 18 10:33:17 sov07l kernel: [1470365.929077] kvm_handle_exit: Breaking 

out of NMI-blocked state on VCPU 3 after 1 s timeout

Does anybody have an idea what this mean? And what we could do, to make the system stable? resolve this error?

Thanks

Andy

You host cpu does not support NMI injection in VMX (your Intel processor is too old). I can't tell for sure if this is what causes w2k8 to reboot itself, but this is possible. Does w2k8 has some kind of NMI watchdog?

--
Gleb.

It doesn't inject NMIs here.

Can you set up memory dumping on BSODs and run the !analyze command in windbg?

error compiling committee.c: too many arguments to function

Hopefully i've done it right.

See below for the output of windbg:

--
0: kd> !analyze -v
*****************************************************************************
**
*
*
* Bugcheck
Analysis *
*
*
*****************************************************************************
**

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000000, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffffa8007db5797, address which referenced memory

Debugging Details:
------------------

READ_ADDRESS: 0000000000000000

CURRENT_IRQL: 2

FAULTING_IP:
+0
fffffa80`07db5797 ?? ???

DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT

BUGCHECK_STR: 0xD1

PROCESS_NAME: System

TRAP_FRAME: fffff80003e7f610 -- (.trap 0xfffff80003e7f610)
NOTE: The trap frame does not contain all registers. Some register values may be zeroed or incorrect. rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000201 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000 rip=fffffa8007db5797 rsp=fffff80003e7f7a0 rbp=fffff80003e7f800
r8=000000000000082f r9=000000000000000d r10=0000000000000000
r11=fffff80001a45640 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na po nc
fffffa80`07db5797 ?? ???
Resetting default scope

LAST_CONTROL_TRANSFER: from fffff800018703ee to fffff80001870650

STACK_TEXT:
fffff80003e7f4c8 fffff800018703ee : 000000000000000a 0000000000000000 0000000000000002 0000000000000000 : nt!KeBugCheckEx fffff80003e7f4d0 fffff8000186f2cb : 0000000000000000 fffffa8004a7d000 fffffa8007a28d00 fffffa8007db418f : nt!KiBugCheckDispatch+0x6e fffff80003e7f610 fffffa8007db5797 : fffff80003e7f7b0 fffff80001d43cb1 0000000000000000 fffffa6000e5c12d : nt!KiPageFault+0x20b fffff80003e7f7a0 fffff80003e7f7b0 : fffff80001d43cb1 0000000000000000 fffffa6000e5c12d fffffa800573bbb0 : 0xfffffa8007db5797 fffff80003e7f7a8 fffff80001d43cb1 : 0000000000000000 fffffa6000e5c12d fffffa800573bbb0 fffffa60005ec180 : 0xfffff80003e7f7b0 fffff80003e7f7b0 fffff80001877005 : fffffa800768a510 fffffa60005ec180 fffffa600085e110 fffffa80050e6340 : hal!HalpRequestIpiSpecifyVector+0x81 fffff80003e7f7e0 fffff80001876773 : ffffffff00000001 fffff80001990680 0000000000000000 0000000000000002 : nt!KiDeferredReadyThread+0x405 fffff80003e7f830 00000000fffffa80 : 01868e030010e380 00000000fffff800 0000000000000000 0000000000000000 : nt!KeSetEvent+0x1f3 fffff80003e7f8a0 01868e030010e380 : 00000000fffff800 0000000000000000 0000000000000000 0000000000000000 : 0xfffffa80 fffff80003e7f8a8 00000000fffff800 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : 0x1868e030010e380 fffff80003e7f8b0 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 00000000`00000000 : 0xfffff800

STACK_COMMAND: kb

FOLLOWUP_IP:
nt!KiPageFault+20b
fffff800`0186f2cb 488d058e320000 lea rax,[nt!
RtlInterlockedPopEntrySList (fffff800`01872560)]

SYMBOL_STACK_INDEX: 2

SYMBOL_NAME: nt!KiPageFault+20b

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: nt

IMAGE_NAME: ntkrnlmp.exe

DEBUG_FLR_IMAGE_TIMESTAMP: 49ac93e1

FAILURE_BUCKET_ID: X64_0xD1_nt!KiPageFault+20b

BUCKET_ID: X64_0xD1_nt!KiPageFault+20b

Followup: MachineOwner
---------

On Thu, 18 Jun 2009 15:45:54 +0300, Avi Kivity wrote

I've found another dump (minidump) witch has some other information (see below):

--

0: kd> !analyze -v
*****************************************************************************
**
*
*
* Bugcheck
Analysis *
*
*
*****************************************************************************
**

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000000, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffffa8007db5797, address which referenced memory

Debugging Details:
------------------

READ_ADDRESS: GetPointerFromAddress: unable to read from fffff80001a45080
0000000000000000

CURRENT_IRQL: 2

FAULTING_IP:
+0
fffffa80`07db5797 ?? ???

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: DRIVER_FAULT_SERVER_MINIDUMP

BUGCHECK_STR: 0xD1

PROCESS_NAME: System

TRAP_FRAME: fffff80003e7f610 -- (.trap 0xfffff80003e7f610)
NOTE: The trap frame does not contain all registers. Some register values may be zeroed or incorrect. rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000201 rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000 rip=fffffa8007db5797 rsp=fffff80003e7f7a0 rbp=fffff80003e7f800
r8=000000000000082f r9=000000000000000d r10=0000000000000000
r11=fffff80001a45640 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na po nc
fffffa80`07db5797 ?? ???
Resetting default scope

LAST_CONTROL_TRANSFER: from fffff800018703ee to fffff80001870650

STACK_TEXT:
fffff80003e7f4c8 fffff800018703ee : 000000000000000a 0000000000000000 0000000000000002 0000000000000000 : nt!KeBugCheckEx fffff80003e7f4d0 fffff8000186f2cb : 0000000000000000 fffffa8004a7d000 fffffa8007a28d00 fffffa8007db418f : nt!KiBugCheckDispatch+0x6e fffff80003e7f610 fffffa8007db5797 : fffff80003e7f7b0 fffff80001d43cb1 0000000000000000 fffffa6000e5c12d : nt!KiPageFault+0x20b fffff80003e7f7a0 fffff80003e7f7b0 : fffff80001d43cb1 0000000000000000 fffffa6000e5c12d fffffa800573bbb0 : 0xfffffa8007db5797 fffff80003e7f7a8 fffff80001d43cb1 : 0000000000000000 fffffa6000e5c12d fffffa800573bbb0 fffffa60005ec180 : 0xfffff80003e7f7b0 fffff80003e7f7b0 fffff80001877005 : fffffa800768a510 fffffa60005ec180 fffffa600085e110 fffffa80050e6340 : hal!HalpRequestIpiSpecifyVector+0x81 fffff80003e7f7e0 fffff80001876773 : ffffffff00000001 fffff80001990680 0000000000000000 0000000000000002 : nt!KiDeferredReadyThread+0x405 fffff80003e7f830 00000000fffffa80 : 01868e030010e380 00000000fffff800 0000000000000000 0000000000000000 : nt!KeSetEvent+0x1f3 fffff80003e7f8a0 01868e030010e380 : 00000000fffff800 0000000000000000 0000000000000000 0000000000000000 : 0xfffffa80 fffff80003e7f8a8 00000000fffff800 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : 0x1868e030010e380 fffff80003e7f8b0 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 00000000`00000000 : 0xfffff800

STACK_COMMAND: kb

FOLLOWUP_IP:
nt!KiPageFault+20b
fffff800`0186f2cb 488d058e320000 lea rax,[nt!
RtlInterlockedPopEntrySList (fffff800`01872560)]

SYMBOL_STACK_INDEX: 2

SYMBOL_NAME: nt!KiPageFault+20b

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: nt

IMAGE_NAME: ntkrnlmp.exe

DEBUG_FLR_IMAGE_TIMESTAMP: 49ac93e1

FAILURE_BUCKET_ID: X64_0xD1_nt!KiPageFault+20b

BUCKET_ID: X64_0xD1_nt!KiPageFault+20b

Followup: MachineOwner

Hi Gleb

Thank you, for fast reply.

On Thu, 18 Jun 2009 14:37:47 +0300, Gleb Natapov wrote

It's a system with 2x L5320 (Quad Core Xeon). (see below)

The reboot it's randomly, sometime during working-hours, but sometimes during the night, when nobody is logged in. Sometimes the System is up over a week, sometimes we have a reboot twice a day.

Don't know what it is, so i don't know.. if it has any.

Thanks

Andy

--

cat /proc/cpuinfo:

processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU L5320 @ 1.86GHz
stepping : 7
cpu MHz : 1866.732
cache size : 4096 KB
physical id : 1
siblings : 4
core id : 3
cpu cores : 4
apicid : 7
initial apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
syscall nx lm
constant
_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl
vmx tm2 ssse3 cx16 xtpr
d
ca lahf_lm
bogomips : 3733.58
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

Is this the only VM that runs on this host? If not what other guests are you running. May be the messages you see are not generated by the problematic guest after all.

--
Gleb.

No, there are about 15 VMs on this Host.

There are Vista, W2k3, w2k8 and debian linuxes running. I think all OSs are x64-systems.

If I look at the time, when this error shows up in the kernel-log, it matches exactly the time, when the machine is rebooting.

Currently I try to use the windbg-tool.. but no success so far..