Houston, we have May 15, 1953 (says guest when host uses cpufreq, and dies)
Is using cpufreq (i.e. with ondemand governor) on KVM host safe for guests?
I enabled cpufreq on the host, it scaled down the host CPU (Dual-Core AMD Opteron(tm) Processor 2212) to 1 GHz from 2 GHz.
Guest (using 1 CPU) was still showing that it has a 2 GHz CPU in /proc/cpuinfo (I guess this value is read only once, when booting).
After about 2 hours I started "date" on the guest - it showed that it's year *1953*, after which I couldn't start any other command (the guest was technically alive - SSH connection to it didn't die - but I couldn't do anything).
# date Wed Feb 18 13:07:17 CET 2009
[let's wait ~2 hours]
# date Fri May 15 10:13:14 CET 1953 # date ^C^Z [could not interrupt]
Is it expected behaviour? Is it correct behaviour?
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 3 lines · Tomasz Chmielewski
Is using cpufreq (i.e. with ondemand governor) on KVM host safe for guests?
I enabled cpufreq on the host, it scaled down the host CPU (Dual-Core AMD Opteron(tm) Processor 2212) to 1 GHz from 2 GHz.
Not with your processor. Intel processors should be fine and any AMD processor that's Barcelona/Phenom or newer.
Regards,
Anthony Liguori
Quoted 20 lines
Guest (using 1 CPU) was still showing that it has a 2 GHz CPU in
/proc/cpuinfo(I guess this value is read only once, when booting).After about 2 hours I started "date" on the guest - it showed that it's year *1953*, after which I couldn't start any other command (the guest was technically alive - SSH connection to it didn't die - but I couldn't do anything).
# date Wed Feb 18 13:07:17 CET 2009[let's wait ~2 hours]
# date Fri May 15 10:13:14 CET 1953 # date ^C^Z [could not interrupt]Is it expected behaviour? Is it correct behaviour?
Quoted 6 lines · Anthony Liguori
Quoted 3 lines · Tomasz Chmielewski
Is using cpufreq (i.e. with ondemand governor) on KVM host safe for guests?
I enabled cpufreq on the host, it scaled down the host CPU (Dual-Core AMD Opteron(tm) Processor 2212) to 1 GHz from 2 GHz.
Not with your processor. Intel processors should be fine and any AMD processor that's Barcelona/Phenom or newer.
Looks I'm a bad, bad, anti-environment CO2 contributor then.
From a technical perspective, what are the problems with my CPU that it scales down on the host just fine, but makes the guests "return to the past"?
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 3 lines · Tomasz Chmielewski
Looks I'm a bad, bad, anti-environment CO2 contributor then.
From a technical perspective, what are the problems with my CPU that it scales down on the host just fine, but makes the guests "return to the past"?
What kvm version are you using? kvm-84 should fix this.
Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.
Quoted 6 lines · Avi Kivity
Quoted 3 lines · Tomasz Chmielewski
Looks I'm a bad, bad, anti-environment CO2 contributor then.
From a technical perspective, what are the problems with my CPU that it scales down on the host just fine, but makes the guests "return to the past"?
What kvm version are you using? kvm-84 should fix this.
I still have kvm-83.
Thanks for the hint, I'll see how it works.
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 6 lines · Avi Kivity
Quoted 3 lines · Tomasz Chmielewski
Looks I'm a bad, bad, anti-environment CO2 contributor then.
From a technical perspective, what are the problems with my CPU that it scales down on the host just fine, but makes the guests "return to the past"?
What kvm version are you using? kvm-84 should fix this.
Are you suggesting that one should use cpufreq on a CPU without a constant tsc? Isn't this just asking for trouble?
Regards,
Anthony Liguori
Quoted 2 lines · Anthony Liguori
Are you suggesting that one should use cpufreq on a CPU without a constant tsc? Isn't this just asking for trouble?
Depends on the (guest) clock source ;)
tsc isn't going to do well obviously.
kvmclock is designed to handle tsc frequency changes just fine. And with the kvm-84 kernel module it actually works correctly.
HTH,
Gerd
Quoted 10 lines · Gerd Hoffmann
Quoted 2 lines · Anthony Liguori
Are you suggesting that one should use cpufreq on a CPU without a constant tsc? Isn't this just asking for trouble?
Depends on the (guest) clock source ;)
tsc isn't going to do well obviously.
kvmclock is designed to handle tsc frequency changes just fine. And with the kvm-84 kernel module it actually works correctly.
So with Linux virtio guests I may have luck, but not so with Windows, which can't (yet?) use kvm-clock. Correct? (it may be some time before I'm able to upgrade and check how it really works).
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 2 lines · Tomasz Chmielewski
So with Linux virtio guests I may have luck, but not so with Windows, which can't (yet?) use kvm-clock. Correct?
tsc isn't the only clocksource, there are also hpet and acpi (pm timer), they shouldn't have trouble with tsc freq changes. Dunno what windows uses by default.
cheers,
Gerd
Quoted 19 lines · Tomasz Chmielewski
Is using cpufreq (i.e. with ondemand governor) on KVM host safe for guests?
I enabled cpufreq on the host, it scaled down the host CPU (Dual-Core AMD Opteron(tm) Processor 2212) to 1 GHz from 2 GHz.
Guest (using 1 CPU) was still showing that it has a 2 GHz CPU in
/proc/cpuinfo(I guess this value is read only once, when booting).After about 2 hours I started "date" on the guest - it showed that it's year *1953*, after which I couldn't start any other command (the guest was technically alive - SSH connection to it didn't die - but I couldn't do anything).
# date Wed Feb 18 13:07:17 CET 2009[let's wait ~2 hours]
Quoted 8 lines
# date Fri May 15 10:13:14 CET 1953 # date ^C^Z [could not interrupt]Is it expected behaviour? Is it correct behaviour?
Whats the output of /proc/cpuinfo on the host? Does it contain the constant_tsc flag?
Whats the output of /sys/devices/system/clocksource/clocksource0/current_clocksource on the guest?
Quoted 22 lines · Marcelo Tosatti
Quoted 21 lines · Tomasz Chmielewski
Is using cpufreq (i.e. with ondemand governor) on KVM host safe for guests?
I enabled cpufreq on the host, it scaled down the host CPU (Dual-Core AMD Opteron(tm) Processor 2212) to 1 GHz from 2 GHz.
Guest (using 1 CPU) was still showing that it has a 2 GHz CPU in
/proc/cpuinfo(I guess this value is read only once, when booting).After about 2 hours I started "date" on the guest - it showed that it's year *1953*, after which I couldn't start any other command (the guest was technically alive - SSH connection to it didn't die - but I couldn't do anything).
# date Wed Feb 18 13:07:17 CET 2009[let's wait ~2 hours]
# date Fri May 15 10:13:14 CET 1953 # date ^C^Z [could not interrupt]
Quoted 4 lines
Quoted 1 line
Is it expected behaviour? Is it correct behaviour?
Whats the output of
/proc/cpuinfoon the host? Does it contain theconstant_tscflag?
It doesn't contain this flag./proc/cpuinfo output - below.
Quoted 3 lines
Whats the output of
/sys/devices/system/clocksource/clocksource0/current_clocksourceon the guest?
# cat /sys/devices/system/clocksource/clocksource0/* hpet acpi_pm jiffies tsc <- available
hpet <- current
# cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 15 model : 65 model name : Dual-Core AMD Opteron(tm) Processor 2212 stepping : 2 cpu MHz : 2000.000 cache size : 1024 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow rep_good pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy bogomips : 3993.20 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: ts fid vid ttp tm stc
processor : 1vendor_id : AuthenticAMD
cpu family : 15 model : 65 model name : Dual-Core AMD Opteron(tm) Processor 2212
stepping : 2 cpu MHz : 2000.000 cache size : 1024 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow rep_good pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy bogomips : 3993.20 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: ts fid vid ttp tm stc
processor : 2vendor_id : AuthenticAMD
cpu family : 15 model : 65 model name : Dual-Core AMD Opteron(tm) Processor 2212
stepping : 2 cpu MHz : 2000.000 cache size : 1024 KB
physical id : 1
siblings : 2
core id : 0
cpu cores : 2
fpu : yesfpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow rep_good pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy
bogomips : 3993.20
TLB size : 1024 4K pages
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc
processor : 3vendor_id : AuthenticAMD
cpu family : 15
model : 65
model name : Dual-Core AMD Opteron(tm) Processor 2212
stepping : 2
cpu MHz : 2000.000
cache size : 1024 KB
physical id : 1
siblings : 2
core id : 1
cpu cores : 2
fpu : yesfpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow rep_good pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy
bogomips : 3993.20
TLB size : 1024 4K pages
clflush size : 64cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 13 lines · Tomasz Chmielewski
processor : 2
vendor_id: AuthenticAMD
cpu family : 15 model : 65 model name
: Dual-Core AMD Opteron(tm) Processor 2212
stepping : 2 cpu MHz
: 2000.000 cache size : 1024 KB
physical id : 1
siblings : 2
core id : 0
cpu cores : 2
fpu : yesfpu_exception: yes
cpuid level : 1
Quoted 8 lines
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxextfxsr_optrdtscp lm 3dnowext 3dnowrep_goodpni cx16lahf_lmcmp_legacysvm extapiccr8_legacy
bogomips : 3993.20
TLB size : 1024 4K pages
clflush size : 64cache_alignment: 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc
kvm-84 as mentioned. Sorry.
Quoted 9 lines · Marcelo Tosatti
Quoted 7 lines
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext
fxsr_optrdtscp lm 3dnowext 3dnowrep_goodpni cx16lahf_lmcmp_legacysvm extapiccr8_legacy
bogomips : 3993.20
TLB size : 1024 4K pages
clflush size : 64cache_alignment: 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stckvm-84 as mentioned. Sorry.
It's OK as long as it will work.
- will Windows guests work?
- what CPU frequency will the guests show? Current host frequency? Host frequency from the moment the guest booted (i.e. right now the guest will show 1GHz even if the host is running at 2GHz, or the way around)?
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 10 lines · Tomasz Chmielewski
Quoted 8 lines · Marcelo Tosatti
Quoted 7 lines
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext
fxsr_optrdtscp lm 3dnowext 3dnowrep_goodpni cx16lahf_lmcmp_legacysvm extapiccr8_legacy
bogomips : 3993.20
TLB size : 1024 4K pages
clflush size : 64cache_alignment: 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc
Quoted 5 lines
Quoted 1 line
kvm-84 as mentioned. Sorry.
It's OK as long as it will work.
- will Windows guests work?
They should. If they don't, please report.
Quoted 3 lines
- what CPU frequency will the guests show? Current host frequency? Host frequency from the moment the guest booted (i.e. right now the guest will show 1GHz even if the host is running at 2GHz, or the way around)?
Host frequency from the moment the guest booted, since the guest does not receive frequency change notifications.
Quoted 5 lines · Marcelo Tosatti
Quoted 2 lines
- what CPU frequency will the guests show? Current host frequency? Host
frequency from the moment the guest booted (i.e. right now the guest will show 1GHz even if the host is running at 2GHz, or the way around)?Host frequency from the moment the guest booted, since the guest does not receive frequency change notifications.
Is it possible (or is it planned) to pass frequency to the guest (the one which is displayed in /proc/cpuinfo)?
Someone may feel disappointed to see his/her brand new virtual guest has a CPU with so few MHz advertised in /proc/cpuinfo.
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 7 lines · Tomasz Chmielewski
Quoted 5 lines · Marcelo Tosatti
Quoted 4 lines
- what CPU frequency will the guests show? Current host frequency? Host frequency from the moment the guest booted (i.e. right now the guest will show 1GHz even if the host is running at 2GHz, or the way around)?
Quoted 5 lines
Quoted 2 lines
Host frequency from the moment the guest booted, since the guest does not receive frequency change notifications.
Is it possible (or is it planned) to pass frequency to the guest (the one which is displayed in
/proc/cpuinfo)?
Possible, not planned AFAIK.
Quoted 2 lines
Someone may feel disappointed to see his/her brand new virtual guest has a CPU with so few MHz advertised in
/proc/cpuinfo.
Thats a point.
Quoted 10 lines · Marcelo Tosatti
Quoted 7 lines · Tomasz Chmielewski
Quoted 4 lines · Marcelo Tosatti
Quoted 2 lines
- what CPU frequency will the guests show? Current host frequency?
Host frequency from the moment the guest booted (i.e. right now the guest will show 1GHz even if the host is running at 2GHz, or the way around)?Host frequency from the moment the guest booted, since the guest does not receive frequency change notifications.
Is it possible (or is it planned) to pass frequency to the guest (the one which is displayed in
/proc/cpuinfo)?Possible, not planned AFAIK.
Possible, right now? How?
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 8 lines · Tomasz Chmielewski
Quoted 7 lines · Marcelo Tosatti
Quoted 6 lines · Tomasz Chmielewski
Quoted 4 lines · Marcelo Tosatti
Quoted 4 lines
- what CPU frequency will the guests show? Current host frequency? Host frequency from the moment the guest booted (i.e. right now the guest will show 1GHz even if the host is running at 2GHz, or the way around)?
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Host frequency from the moment the guest booted, since the guest does not receive frequency change notifications.
Is it possible (or is it planned) to pass frequency to the guest (the
one which is displayed in/proc/cpuinfo)?Possible, not planned AFAIK.
Possible, right now? How?
Write a paravirt notification scheme.
Quoted 7 lines · Marcelo Tosatti
Quoted 6 lines · Tomasz Chmielewski
Quoted 5 lines · Marcelo Tosatti
Quoted 4 lines · Tomasz Chmielewski
Quoted 2 lines · Marcelo Tosatti
Quoted 2 lines
- what CPU frequency will the guests show? Current host
frequency? Host frequency from the moment the guest booted (i.e. right now the guest will show 1GHz even if the host is running at 2GHz, or the way around)?
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines
Host frequency from the moment the guest booted, since the guest does not receive frequency change notifications.
Is it possible (or is it planned) to pass frequency to the guest (the
one which is displayed in/proc/cpuinfo)?Possible, not planned AFAIK.
Possible, right now? How?
Write a paravirt notification scheme.
That's a bit low level.
I was thinking of a parameter to kvm (binary) which would pass the value to the guest.
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Quoted 10 lines · Marcelo Tosatti
Quoted 7 lines
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext
fxsr_optrdtscp lm 3dnowext 3dnowrep_goodpni cx16lahf_lmcmp_legacysvm extapiccr8_legacy
bogomips : 3993.20
TLB size : 1024 4K pages
clflush size : 64cache_alignment: 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stckvm-84 as mentioned. Sorry.
It is not stable for me (although I did go through exactly the same routine with kvm-83) - I got an oops. I'm not sure if the problem is kvm or cpufreq.
This is what I did:
- stopped all kvm-83 guests, removed all kvm-83 modules
- inserted kvm-83 modules, started 9 guests with kvm-84 binary
- inserted cpufreq modules and set the governor to ondemand
- cat /proc/cpuinfo revealed that CPUs are still running at full speed, so I did:
cat /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
but it didn't return and cat was in "D" state.
- I stopped all guests, removed kvm-84 modules, and got this oops:
Unable to handle kernel paging request at ffffffff88429721 RIP: [<ffffffff88429721>] PGD 203067 PUD 207063 PMD 1184dc067 PTE 0 Oops: 0010 [1] PREEMPT SMP CPU: 0 Modules linked in: loop cpufreq_ondemand powernow_k8 freq_table crc32c libcrc32c vzethdev vznetdev simfs vzrst vzcpt tun vzdquota vzmon ipv6 vzdev xt_tcpudp xt_length ipt_ttl xt_tcpmss xt_TCPMSS iptable_mangle iptable_filter xt_multiport xt_limit ipt_tos ipt_REJECT ip_tables x_tables ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi scsi_transport_iscsi bridge 8021q bonding dm_snapshot dm_mirror dm_multipath dm_mod joydev usbhid hid sata_svw pata_serverworks psmouse ehci_hcd ohci_hcd evdev thermal button ata_generic pata_acpi serio_raw i2c_piix4 tg3 container pcspkr libata usbcore processor i2c_core ssb shpchp pci_hotplug k8temp romfs isofs sd_mod sg mptsas mptscsih mptbase scsi_transport_sas scsi_mod raid1 md_mod Pid: 20389, comm: kondemand/0 Not tainted 2.6.24-2-pve #1 ovz005 RIP: 0010:[<ffffffff88429721>] [<ffffffff88429721>] RSP: 0018:ffff810039d09d20 EFLAGS: 00010202 RAX: 0000000000000001 RBX: 00000000ef41d1f5 RCX: ffffffff806cf690 RDX: 0000000000000000 RSI: ffff810080959000 RDI: ffffffff88450e80 RBP: 0000000000000000 R08: 00000000001e8480 R09: 0000000000000000 R10: ffff810001027ee0 R11: 0000000000000001 R12: ffffffff88450e90 R13: ffff810039d09df0 R14: 0000000000000000 R15: 0000000000000000 FS: 00007f93a7d736d0(0000) GS:ffffffff8060b000(0000) knlGS:0000000000000000 CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b CR2: ffffffff88429721 CR3: 0000000000201000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process kondemand/0 (pid: 20389, veid=0, threadinfo ffff810039d08000, task ffff810039d06000) Stack: 00000000ef41d1f5 0000000000000000 ffffffff88451950 ffff810039d09df0 0000000000000001 ffffffff804aa671 ffffffff8077b1f0 ffff810039d09df0 0000000000000000 00000000ffffffff ffffffff8077b1c0 ffffffff8025ca0a Call Trace: [<ffffffff804aa671>] notifier_call_chain+0x51/0x70 [<ffffffff8025ca0a>] __srcu_notifier_call_chain+0x5a/0x90 [<ffffffff8040897d>] cpufreq_notify_transition+0x7d/0xb0 [<ffffffff8846f9ef>] :powernow_k8:powernowk8_target+0x2bf/0x690 [<ffffffff8847671a>] :cpufreq_ondemand:do_dbs_timer+0x26a/0x300 [<ffffffff884764b0>] :cpufreq_ondemand:do_dbs_timer+0x0/0x300 [<ffffffff80252a38>] run_workqueue+0x88/0x120 [<ffffffff80253610>] worker_thread+0x0/0x130 [<ffffffff802536d5>] worker_thread+0xc5/0x130 [<ffffffff80257f70>] autoremove_wake_function+0x0/0x30 [<ffffffff80253610>] worker_thread+0x0/0x130 [<ffffffff80253610>] worker_thread+0x0/0x130 [<ffffffff80257b9b>] kthread+0x4b/0x80 [<ffffffff8020d338>] child_rip+0xa/0x12 [<ffffffff80257b50>] kthread+0x0/0x80 [<ffffffff8020d32e>] child_rip+0x0/0x12 Code: Bad RIP value. RIP [<ffffffff88429721>] RSP <ffff810039d09d20> CR2: ffffffff88429721 ---[ end trace e6b0e16fe814aeb1 ]---
note: kondemand/0[20389] exited with preempt_count 1
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org