Match virtual machine processor vendor to that of the host

PATCH In review Jeremy Katz 3 participants 8 messages Dec 21, 2006 – Dec 24, 2006 6 messages not in the archive KVM

Share & tools

Jeremy Katz

author

2006-12-21 04:34 UTC #0 lore raw

PATCH 1/1 Match virtual machine processor vendor to that of the host

Currently, kvm ends up just using the standard qemu cpu initialization. This means that all x86_64 virtual machines appear to have an AuthenticAMD (AMD64) processor. This ends up causing a problem when booting some x86_64 Linux kernels as they attempt to do AMD64 specific initialization for things like performance counters. Since those MSRs aren't supported on the Intel host, the virtual machine is halted.

The attached patch makes it so that we check the actual cpu type of the host and then provide the same cpu type for the virtual machine.

Jeremy

exec-all.h +4 -0

Index: qemu/exec-all.h
===================================================================
@@ -603,3 +603,7 @@
 }
 
 #endif
+
+#ifdef USE_KVM
+#include "qemu-kvm.h"
+#endif

target-i386/helper2.c +3 -0

@@ -141,6 +141,9 @@
 #ifdef USE_KQEMU
     kqemu_init(env);
 #endif
+#ifdef USE_KVM
+    kvm_cpu_init(env);
+#endif
     return env;
 }

qemu-kvm.c +50 -0

56 lines not on this page · raw message

qemu-kvm.h +2 -0

Index: qemu/qemu-kvm.h
===================================================================
@@ -10,4 +10,6 @@
 int kvm_cpu_exec(CPUState *env);
 int kvm_update_debugger(CPUState *env);
 
+int kvm_cpu_init(CPUState *env);
+
 #endif

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

A message that is not in the archive (sent off-list or not yet received): 1166675645.10717.99.camel-T9xAYgMuJli44ywRPIzf9A@public.gmane.org

Avi Kivity

2006-12-21 09:20 UTC #2 lore raw

Quoted 11 lines · Jeremy Katz

Currently, kvm ends up just using the standard qemu cpu initialization. This means that all x86_64 virtual machines appear to have an AuthenticAMD (AMD64) processor. This ends up causing a problem when booting some x86_64 Linux kernels as they attempt to do AMD64 specific initialization for things like performance counters. Since those MSRs aren't supported on the Intel host, the virtual machine is halted.
The attached patch makes it so that we check the actual cpu type of the host and then provide the same cpu type for the virtual machine.

Windows crashes immediately with this patch (STOP: 0x7E (0xc0000005 0xF724879C 0xF7A168DC 0xF7A165D8) for those who know how to decode BSODs), so either it's doing too much or not enough.

Note that we support migration from Intel to AMD and vice versa, so it would be good to supply a command line parameter to set the cpuid (to a least common denominator CPU).

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 458A51C0.6020506-atKUWr5tajBWk0Htik3J/[email protected]

Michael Riepe

2006-12-21 09:49 UTC #4 lore raw

Hi!

Quoted 7 lines · Avi Kivity

Windows crashes immediately with this patch (STOP: 0x7E (0xc0000005 0xF724879C 0xF7A168DC 0xF7A165D8) for those who know how to decode BSODs), so either it's doing too much or not enough.
Note that we support migration from Intel to AMD and vice versa, so it would be good to supply a command line parameter to set the cpuid (to a least common denominator CPU).

That would be extremely important for live migration. I doubt that any current OS - even those that support CPU hot-plugging - will allow you to replace an AMD CPU with an Intel model (or vice versa) on the fly.

Signature · 8 lines

Michael "Tired" Riepe <[email protected]>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Jeremy Katz

author

2006-12-21 14:01 UTC #5 lore raw

In reply to a message not in the archive

Quoted 8 lines · Avi Kivity

Quoted 7 lines · Jeremy Katz
Currently, kvm ends up just using the standard qemu cpu initialization. This means that all x86_64 virtual machines appear to have an AuthenticAMD (AMD64) processor. This ends up causing a problem when booting some x86_64 Linux kernels as they attempt to do AMD64 specific initialization for things like performance counters. Since those MSRs aren't supported on the Intel host, the virtual machine is halted.

Quoted 6 lines

Quoted 2 lines
The attached patch makes it so that we check the actual cpu type of the host and then provide the same cpu type for the virtual machine.
Windows crashes immediately with this patch (STOP: 0x7E (0xc0000005 0xF724879C 0xF7A168DC 0xF7A165D8) for those who know how to decode BSODs), so either it's doing too much or not enough.

32 or 64 bit and what are you running for the host? There's definitely more that _could_ be mirrored.

Quoted 3 lines

Note that we support migration from Intel to AMD and vice versa, so it would be good to supply a command line parameter to set the cpuid (to a least common denominator CPU).

The problem is that with current CPUs, I don't know that there really is a least common denominator anymore. Both Intel and AMD have their own implementation of a number of things which aren't "core" bits -- performance counters, things to handle microcode updates, etc. :-/

Are there any docs for the current state of migration? Depending on how it's done, this may not at least make things any "worse" for the migration case since we're only changing things at CPU startup, not runtime calls to cpuid.

Jeremy

A message that is not in the archive (sent off-list or not yet received): 1166709696.10717.113.camel-T9xAYgMuJli44ywRPIzf9A@public.gmane.org

Avi Kivity

2006-12-21 14:26 UTC #7 lore raw

Quoted 14 lines · Jeremy Katz

Quoted 12 lines · Avi Kivity
Quoted 10 lines · Jeremy Katz
Currently, kvm ends up just using the standard qemu cpu initialization. This means that all x86_64 virtual machines appear to have an AuthenticAMD (AMD64) processor. This ends up causing a problem when booting some x86_64 Linux kernels as they attempt to do AMD64 specific initialization for things like performance counters. Since those MSRs aren't supported on the Intel host, the virtual machine is halted.
The attached patch makes it so that we check the actual cpu type of the host and then provide the same cpu type for the virtual machine.

Quoted 9 lines

Quoted 4 lines
Windows crashes immediately with this patch (STOP: 0x7E (0xc0000005 0xF724879C 0xF7A168DC 0xF7A165D8) for those who know how to decode BSODs), so either it's doing too much or not enough.
32 or 64 bit and what are you running for the host? There's definitely more that _could_ be mirrored.

Host 64-bit Intel Pentium D; guest is 32-bit Windows XP Pro (64-bit Windows depends on the next qemu merge).

Quoted 11 lines

Quoted 4 lines
Note that we support migration from Intel to AMD and vice versa, so it would be good to supply a command line parameter to set the cpuid (to a least common denominator CPU).
The problem is that with current CPUs, I don't know that there really is a least common denominator anymore. Both Intel and AMD have their own implementation of a number of things which aren't "core" bits -- performance counters, things to handle microcode updates, etc. :-/

One way around is to report a non-AMD, non-Intel processor ('FakeKVM' signature?)

But usually guests test for the feature using cpuid, so we just have to turn off a lot of bits there.

Note that qemu manages to run most hosts without emulating all those weird features, so it shouldn't be too hard.

Quoted 2 lines

Are there any docs for the current state of migration? Depending on how it's done,

Migration just moves the cpu state, the hardware state, and memory to another host. Live migration does this in parallel with execution.

What do you mean by "current state of migration"? How to run it for testing?

Quoted 4 lines

this may not at least make things any "worse" for the
migration case since we're only changing things at CPU startup, not runtime calls to cpuid.

If you tell the host, for instance, that you support the AMD performance counters and then you migrate to Intel, things will break. Best to tell the guest that you don't support performance counters at all (this is necessary from an isolation point of view as well).

Having said all that, I think the problem with your patch is just some minor internal inconsistency that causes Windows to barf. Decoding the oops from the microsoft docs will probably shed some light on the matter.

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 458A99AB.30701-atKUWr5tajBWk0Htik3J/[email protected]

Jeremy Katz

author

2006-12-21 15:49 UTC #9 lore raw

Quoted 28 lines · Avi Kivity

Quoted 12 lines · Jeremy Katz
Quoted 11 lines · Avi Kivity
Quoted 10 lines · Jeremy Katz
Currently, kvm ends up just using the standard qemu cpu initialization. This means that all x86_64 virtual machines appear to have an AuthenticAMD (AMD64) processor. This ends up causing a problem when booting some x86_64 Linux kernels as they attempt to do AMD64 specific initialization for things like performance counters. Since those MSRs aren't supported on the Intel host, the virtual machine is halted.
The attached patch makes it so that we check the actual cpu type of the host and then provide the same cpu type for the virtual machine.
Quoted 7 lines
Quoted 4 lines
Windows crashes immediately with this patch (STOP: 0x7E (0xc0000005 0xF724879C 0xF7A168DC 0xF7A165D8) for those who know how to decode BSODs), so either it's doing too much or not enough.
32 or 64 bit and what are you running for the host? There's definitely more that _could_ be mirrored.
Host 64-bit Intel Pentium D; guest is 32-bit Windows XP Pro (64-bit Windows depends on the next qemu merge).
Quoted 4 lines
Quoted 3 lines
Note that we support migration from Intel to AMD and vice versa, so it would be good to supply a command line parameter to set the cpuid (to a least common denominator CPU).

Quoted 8 lines

Quoted 5 lines
The problem is that with current CPUs, I don't know that there really is a least common denominator anymore. Both Intel and AMD have their own implementation of a number of things which aren't "core" bits -- performance counters, things to handle microcode updates, etc. :-/
One way around is to report a non-AMD, non-Intel processor ('FakeKVM' signature?)

I'm not sure how different OS's will actually respond to that, either... I have a sinking suspicion that it would definitely throw some for a loop based on prior experiences with a new processor rollout :/

Quoted 2 lines

But usually guests test for the feature using cpuid, so we just have to turn off a lot of bits there.

There are a lot of things which aren't exposed as explicit features via cpuid that an OS can and will care about. Using cpuid to mask sse3 or the like should be fine, but really doesn't take care of the breadth of what's needed.

Quoted 2 lines

Note that qemu manages to run most hosts without emulating all those weird features, so it shouldn't be too hard.

For i386 qemu, it emulates a processor before a lot of the vendor-specific features were really available (it looks like it pretends to be a PentiumII). x86_64 is much more of a state where there are differences between the Intel and AMD chips and the "right" thing is switched at runtime.

Quoted 8 lines

Quoted 2 lines
Are there any docs for the current state of migration? Depending on how it's done,
Migration just moves the cpu state, the hardware state, and memory to another host. Live migration does this in parallel with execution.
What do you mean by "current state of migration"? How to run it for testing?

Yep

Quoted 9 lines

Quoted 4 lines
this may not at least make things any "worse" for the
migration case since we're only changing things at CPU startup, not runtime calls to cpuid.
If you tell the host, for instance, that you support the AMD performance counters and then you migrate to Intel, things will break. Best to tell the guest that you don't support performance counters at all (this is necessary from an isolation point of view as well).

The only thing that keys supporting the specific processor features/msrs, though, is the vendor and model. eg, see arch/x86_64/kernel/nmi.c:setup_apic_nmi_watchdog(). It's not only things that are expressed specifically from the cpuid (eg, mmx or sse)

Quoted 3 lines

Having said all that, I think the problem with your patch is just some minor internal inconsistency that causes Windows to barf. Decoding the oops from the microsoft docs will probably shed some light on the matter.

Indeed -- I'll try to find a Windows CD so that I can take a look this afternoon.

Jeremy

A message that is not in the archive (sent off-list or not yet received): 1166716174.10717.145.camel-T9xAYgMuJli44ywRPIzf9A@public.gmane.org

Avi Kivity

2006-12-21 16:16 UTC #11 lore raw

Quoted 6 lines · Jeremy Katz

Quoted 3 lines
What do you mean by "current state of migration"? How to run it for testing?
Yep

I've asked Uri to add a section to the web site.

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 458AB35A.3060402-atKUWr5tajBWk0Htik3J/[email protected]

Avi Kivity

2006-12-24 10:54 UTC #13 lore raw

Quoted 10 lines · Avi Kivity

Quoted 6 lines · Jeremy Katz
Quoted 3 lines
What do you mean by "current state of migration"? How to run it for testing?
Yep
I've asked Uri to add a section to the web site.

This is now http://kvm.sourceforge.net/migration.html.

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV