Skip to contentSkip to conversation

broken timer

DISCUSSION Glauber Costa 2 participants 3 messages Jul 27, 2009 – Jul 28, 2009 KVM

lore
Share & tools
lore
#0 lore raw

Hello, goodfellas

I'm seeing a strange problem in our much loved qemu-kvm.git It's been there before avi left for vacation, at least.

The worst part, is that it doesn't happen always, and I don't even think it is deterministic in its nature, IOW, there was nothing I could do to make it more or less likely to happen.

It's almost obviously interrupt related, but I can't determine more than that As I haven't, and won't have the time to debug this in the near future, here's the riddle for you all to appreciate:

..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: QEMU Virtual CPU version 0.10.50 stepping 03 Using local APIC timer interrupts.
Detected 0.000 MHz APIC timer.

------------[ cut here ]------------                                                                                                                         
WARNING: at kernel/time/clockevents.c:46 clockevent_delta2ns+0x37/0x72() (Not tainted)                                                                       
Modules linked in:                                                                                                                                           

Pid: 1, comm: swapper Not tainted 2.6.27.5-117.fc10.x86_64 #1

Call Trace:
 [<ffffffff810418f2>] warn_on_slowpath+0x60/0x90
 [<ffffffff81331c7e>] ? trace_hardirqs_on_thunk+0x3a/0x3c
 [<ffffffff8159d140>] ? early_idt_handler+0x0/0x72       
 [<ffffffff8132faf6>] ? printk+0x3c/0x3e                 
 [<ffffffff8105cb9a>] clockevent_delta2ns+0x37/0x72      
 [<ffffffff815ad263>] setup_boot_APIC_clock+0x1c2/0x24b  
 [<ffffffff8132faf6>] ? printk+0x3c/0x3e                 
 [<ffffffff815ab8a6>] native_smp_prepare_cpus+0x29e/0x2cf
 [<ffffffff8159d5a8>] kernel_init+0x59/0x214             
 [<ffffffff81331c7e>] ? trace_hardirqs_on_thunk+0x3a/0x3c
 [<ffffffff810116e9>] child_rip+0xa/0x11                 
 [<ffffffff81010a07>] ? restore_args+0x0/0x30            
 [<ffffffff8159d54f>] ? kernel_init+0x0/0x214            
 [<ffffffff810116df>] ? child_rip+0x0/0x11

---[ end trace 4eaa2a86a8e2da22 ]---

APIC frequency too slow, disabling apic timer

#1 lore raw
Quoted 3 lines · Glauber Costa

Hello, goodfellas

I'm seeing a strange problem in our much loved qemu-kvm.git

This bug shouldn't depend on qemu-kvm.git at all unless you are running with no-kvm-irqchip. The only things that involved in APIC timer calibration are tsc and APIC. (If you don't use apicpmtimer kernel parameter. Don't you?)

What is you host HW? Which version of kernel modules are you using? Is your host overcommitted when this happens? Try to load the host with work (while(1)) and run the guest. Is it easier to reproduce problem this way?

Quoted 36 lines

It's been there before avi left for vacation, at least.

The worst part, is that it doesn't happen always, and I don't even think it is deterministic in its nature, IOW, there was nothing I could do to make it more or less likely to happen.

It's almost obviously interrupt related, but I can't determine more than that As I haven't, and won't have the time to debug this in the near future, here's the riddle for you all to appreciate:

..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: QEMU Virtual CPU version 0.10.50 stepping 03 Using local APIC timer interrupts.
Detected 0.000 MHz APIC timer.

------------[ cut here ]------------                                                                                                                         
WARNING: at kernel/time/clockevents.c:46 clockevent_delta2ns+0x37/0x72() (Not tainted)                                                                       
Modules linked in:                                                                                                                                           

Pid: 1, comm: swapper Not tainted 2.6.27.5-117.fc10.x86_64 #1

Call Trace:
 [<ffffffff810418f2>] warn_on_slowpath+0x60/0x90
 [<ffffffff81331c7e>] ? trace_hardirqs_on_thunk+0x3a/0x3c
 [<ffffffff8159d140>] ? early_idt_handler+0x0/0x72       
 [<ffffffff8132faf6>] ? printk+0x3c/0x3e                 
 [<ffffffff8105cb9a>] clockevent_delta2ns+0x37/0x72      
 [<ffffffff815ad263>] setup_boot_APIC_clock+0x1c2/0x24b  
 [<ffffffff8132faf6>] ? printk+0x3c/0x3e                 
 [<ffffffff815ab8a6>] native_smp_prepare_cpus+0x29e/0x2cf
 [<ffffffff8159d5a8>] kernel_init+0x59/0x214             
 [<ffffffff81331c7e>] ? trace_hardirqs_on_thunk+0x3a/0x3c
 [<ffffffff810116e9>] child_rip+0xa/0x11                 
 [<ffffffff81010a07>] ? restore_args+0x0/0x30            
 [<ffffffff8159d54f>] ? kernel_init+0x0/0x214            
 [<ffffffff810116df>] ? child_rip+0x0/0x11

---[ end trace 4eaa2a86a8e2da22 ]---
Quoted 6 lines

APIC frequency too slow, disabling apic timer

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
Gleb.

#2 lore raw
Quoted 9 lines · Gleb Natapov
Quoted 3 lines · Glauber Costa

Hello, goodfellas

I'm seeing a strange problem in our much loved qemu-kvm.git

This bug shouldn't depend on qemu-kvm.git at all unless you are running with no-kvm-irqchip. The only things that involved in APIC timer calibration are tsc and APIC. (If you don't use apicpmtimer kernel parameter. Don't you?)

Quoted 4 lines

What is you host HW? Which version of kernel modules are you using? Is your host overcommitted when this happens? Try to load the host with work (while(1)) and run the guest. Is it easier to reproduce problem this way?

NM.

I did a git pull in kvm.git, and reboot my kernel. Not a single problem since then.