setting time in guest with ntpdate results in VM hang
Hello.
We are running a guest OS of CentOS 4.4 (kernel 2.6.12) for legacy reasons, upgrading is not an option. NTP is running on the host and synching with a local GPS NTP server. But due to frequency drift in the guest it restarts itself periodically and upon start up performs an ntpdate to force a time jump on the guest.
I have seen 2 occasions now (over 2 months) where the VM hangs right as the ntpdate command alters the guest clock (based on output in /var/log/messages).
From the host's perspective the VM is still running but it appears to be using very high CPU percentage (more than typical). The only recovery option is to force shutdown of the VM and restart it.
1. Are there any known issues with ntpdate and VMs hanging? Any workarounds?
2. Are there any debugging tools further characterise the problem?
Thanks in advance,
Dale
Further specifics, let me know if more details are needed:
Host:
# uname -a Linux s1pscu01host 2.6.32-220.el6.x86_64 #1 SMP Sat Dec 10 17:04:11 CST 2011 x86_64 x86_64 x86_64 GNU/Linux
on Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
with qemu-kvm-0.12.1.2-2.209.el6.x86_64
Guest:
I don't use any kernel time modifiers in the guest ... kernel /vmlinuz-2.6.12 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
CentOS 4.4
Quoted 8 lines · Dale Swanston
Hello.
We are running a guest OS of CentOS 4.4 (kernel 2.6.12) for legacy
reasons, upgrading is not an option. NTP is running on the host
and synching with a local GPS NTP server. But due to frequency drift in the guest it restarts itself periodically and upon start up performs an ntpdate to force a time jump on the guest.
Quoted 7 lines
I have seen 2 occasions now (over 2 months) where the VM hangs right as the ntpdate command alters the guest clock (based on output in
/var/log/messages).From the host's perspective the VM is still running but it appears to be using very high CPU percentage (more than typical). The only recovery option is to force shutdown of the VM and restart it.
This should not happen.
Quoted 3 lines
1. Are there any known issues with ntpdate and VMs hanging? Any workarounds?
2. Are there any debugging tools further characterise the problem?
Upgrading the guest kernel is not an option? At least install recent kernel in guest to confirm that its not an already fixed bug.
Quoted 24 lines
Thanks in advance,
DaleFurther specifics, let me know if more details are needed:
Host:
# uname -a Linux s1pscu01host 2.6.32-220.el6.x86_64 #1 SMP Sat Dec 10 17:04:11 CST 2011 x86_64 x86_64 x86_64 GNU/Linuxon Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
with qemu-kvm-0.12.1.2-2.209.el6.x86_64
Guest:
I don't use any kernel time modifiers in the guest ... kernel /vmlinuz-2.6.12 ro root=/dev/VolGroup00/LogVol00 rhgb quietCentOS 4.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 16 lines · Marcelo Tosatti
Quoted 8 lines · Dale Swanston
Hello.
We are running a guest OS of CentOS 4.4 (kernel 2.6.12) for legacy
reasons, upgrading is not an option. NTP is running on the host
and synching with a local GPS NTP server. But due to frequency drift in the guest it restarts itself periodically and upon start up performs an ntpdate to force a time jump on the guest.Quoted 7 lines
I have seen 2 occasions now (over 2 months) where the VM hangs right as the ntpdate command alters the guest clock (based on output in
/var/log/messages).From the host's perspective the VM is still running but it appears
to be using very high CPU percentage (more than typical). The only recovery option is to force shutdown of the VM and restart it.
Quoted 7 lines
This should not happen.
Quoted 3 lines
1. Are there any known issues with ntpdate and VMs hanging? Any workarounds?
2. Are there any debugging tools further characterise the problem?Upgrading the guest kernel is not an option? At least install recent kernel in guest to confirm that its not an already fixed bug.
Good idea. I'll try that.
But are there any tools available to determine what the VM is doing when it appears hung? I've looked but haven't found much on debug or diagnostics on a running VM. Any links?
Is it possible the guest kernel is panicking? What would the VM do if that happened? Would it shutdown?
Thanks again.
Quoted 5 lines · Dale Swanston
Good idea. I'll try that.
But are there any tools available to determine what the VM is doing when it appears hung? I've looked but haven't found much on debug or diagnostics on a running VM. Any links?
If you have the vmlinux, enable the gdbserver stub via Qemu's monitor. Then use 'gdb vmlinux', connect to the VM 'target remote host:port' and look at the backtrace.
I have seen something similar using kvm-clock in a guest running 2.6.27.
David
Quoted 10 lines · David Ahern
Quoted 5 lines · Dale Swanston
Good idea. I'll try that.
But are there any tools available to determine what the VM is doing when it appears hung? I've looked but haven't found much on debug or diagnostics on a running VM. Any links?
If you have the vmlinux, enable the gdbserver stub via Qemu's monitor. Then use 'gdb vmlinux', connect to the VM 'target remote host:port' and look at the backtrace.
Another option is to boot the host with profile=kvm, wait for the guest to hang, then do:
readprofile -r ; readprofile -m System-map-of-guest.map
Quoted 3 lines
I have seen something similar using kvm-clock in a guest running 2.6.27.
David