kvm userspace release 6

ANNOUNCE Applied Avi Kivity 5 participants 14 messages Dec 11, 2006 – Dec 12, 2006 9 messages not in the archive KVM

Share & tools

Avi Kivity

author

2006-12-11 11:37 UTC #0 lore raw

http://kvm.sourceforge.net

Changes:
- AMD 32-bit host support (Anthony Liguori)
- more migration work
- stabilization
- improved performance
- improved guest support on AMD

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Michael Riepe

2006-12-11 17:39 UTC #2 lore raw

In reply to a message not in the archive

Hi!

I immediately tried the latest version (SVN release 4064) on my Thinkpad (Core Duo T2400, Linux 2.6.19) but I still have the same problems I had with release 5.

Suse 10.1 (the German version) dies with:

Quoted 2 lines

exception 13 (0)
Aborted

no matter if I boot from DVD or disk.

OpenSolaris XXL gets hit even worse:

Quoted 7 lines

unhandled vm exit: 0x80000021
rax 0000000060000010 rbx 0000000000002000 rcx 000000000102808c rdx 0000000000101e50 rsi 0000000000004f15 rdi 0000000000000000 rsp 0000000000004ffc rbp 0000000001028064 r8 00000000010113c8 r9 00000000000007ff r10 00000000e0000011 r11 b7f04674b7f03ff4 r12 bfa68a80b7f044c8 r13 b7f04674b7efae3c r14 00000001b79a03f0 r15 0000000000000001 rip 00000000000021ab rflags 00223002
Aborted

This one obviously points to an error on VM entry, according to the Intel docs. Maybe the Solaris bootloader is messing with registers it shouldn't touch?

Windows XP Professional works, but only if I turn off ACPI (-no-acpi). The only thing that fails is poweroff/reboot (well, no ACPI, no way to turn off the power, right?) but you can do that manually. If I turn ACPI on, I get a bluescreen complaining about "IRQL_NOT_LESS_OR_EQUAL", whatever that means.

On the other hand, some systems worked fine out of the box. So far I managed to boot and use Knoppix 5.0 and the latest Qemu-Puppy (2.01-3).

All systems I tested work with -no-kvm (or the regular qemu 0.8.2).

The patch I attached is not essential. It fixes two error messages that occur on a 32-bit host (where the offending registers don't exist).

By the way: I get myriads of "kvm: unhandled rdmsr: c0000081" messages in my syslog, but they don't seem to cause any further harm.

Please let me know if (and how, preferably) I can help debugging.

Signature · 2 lines

Michael "Tired" Riepe <[email protected]>
X-Tired: Each morning I get up I die a little

vmx.c +2 -0

Index: kernel/vmx.c
===================================================================
@@ -1175,8 +1175,10 @@
                                VM_ENTRY_CONTROLS, 0);
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, 0);  /* 22.2.1 */
 
+#ifdef CONFIG_X86_64
 	vmcs_writel(VIRTUAL_APIC_PAGE_ADDR, 0);
 	vmcs_writel(TPR_THRESHOLD, 0);
+#endif
 
 	vmcs_writel(CR0_GUEST_HOST_MASK, KVM_GUEST_CR0_MASK);
 	vmcs_writel(CR4_GUEST_HOST_MASK, KVM_GUEST_CR4_MASK);

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Dor Laor

2006-12-11 22:55 UTC #4 lore raw

In reply to a message not in the archive

Quoted 4 lines

Hi!
I immediately tried the latest version (SVN release 4064) on my

Thinkpad

Quoted 1 line

(Core Duo T2400, Linux 2.6.19) but I still have the same problems I

had

Quoted 8 lines

with release 5.
Suse 10.1 (the German version) dies with:
Quoted 2 lines
exception 13 (0)
Aborted
no matter if I boot from DVD or disk.

Probably some real-mode emulation problem. I'm not absolutely sure, will take a look tomorrow when I'll see the code.
There are real-modes instructions that we might fail to emulate correctly. A temporal workaround is to try switch to textual boot mode, it might help till we fix it.
Please tell if it worked for you.

Quoted 10 lines

OpenSolaris XXL gets hit even worse:
Quoted 2 lines
unhandled vm exit: 0x80000021
rax 0000000060000010 rbx 0000000000002000 rcx 000000000102808c rdx
0000000000101e50
Quoted 1 line
rsi 0000000000004f15 rdi 0000000000000000 rsp 0000000000004ffc rbp
0000000001028064
Quoted 1 line
r8 00000000010113c8 r9 00000000000007ff r10 00000000e0000011 r11
b7f04674b7f03ff4

Quoted 8 lines

Quoted 1 line
r12 bfa68a80b7f044c8 r13 b7f04674b7efae3c r14 00000001b79a03f0 r15
0000000000000001
Quoted 2 lines
rip 00000000000021ab rflags 00223002
Aborted
This one obviously points to an error on VM entry, according to the Intel docs. Maybe the Solaris bootloader is messing with registers it shouldn't touch?

Right about this caused by problem in VM entry.
99% chance Solaris is not the blame (1% they are using the BIG-real-mode).
Might be also a graphical boot loader problem, there is a debug.c file that I need to revive that tells what bothered Intel.
In general, Intel tests to deeply before vmentery.

Quoted 4 lines

Windows XP Professional works, but only if I turn off ACPI (-no-acpi). The only thing that fails is poweroff/reboot (well, no ACPI, no way to turn off the power, right?) but you can do that manually. If I turn

ACPI

Quoted 3 lines

on, I get a bluescreen complaining about "IRQL_NOT_LESS_OR_EQUAL", whatever that means.

KVM has an issue with APIC. Currently the APIC is implemented within Qemu.
I will send a patch that moves the APIC into the kernel that fixes the problem you had above by the end of the week *so stay tuned*. There is a simple workaround detailed in the KVM faq.

Quoted 2 lines

On the other hand, some systems worked fine out of the box. So far I managed to boot and use Knoppix 5.0 and the latest Qemu-Puppy

(2.01-3).

Quoted 4 lines

All systems I tested work with -no-kvm (or the regular qemu 0.8.2).
The patch I attached is not essential. It fixes two error messages

that

Quoted 1 line

occur on a 32-bit host (where the offending registers don't exist).

Thanks, will be very soon embedded.

Quoted 9 lines

By the way: I get myriads of "kvm: unhandled rdmsr: c0000081" messages in my syslog, but they don't seem to cause any further harm.
Please let me know if (and how, preferably) I can help debugging.
--
Michael "Tired" Riepe <[email protected]>
X-Tired: Each morning I get up I die a little

Michael Riepe

2006-12-12 07:27 UTC #6 lore raw

In reply to a message not in the archive

Hi!

Quoted 15 lines · Dor Laor

Quoted 3 lines
Hi!
I immediately tried the latest version (SVN release 4064) on my
Thinkpad
Quoted 1 line
(Core Duo T2400, Linux 2.6.19) but I still have the same problems I
had
Quoted 5 lines
with release 5.
Suse 10.1 (the German version) dies with:

Quoted 8 lines

Quoted 4 lines
Quoted 2 lines
exception 13 (0)
Aborted
no matter if I boot from DVD or disk.
Probably some real-mode emulation problem. I'm not absolutely sure, will take a look tomorrow when I'll see the code.

I suspect that it's caused by a segmentation limit violation.

Quoted 4 lines

There are real-modes instructions that we might fail to emulate correctly. A temporal workaround is to try switch to textual boot mode, it might help till we fix it.
Please tell if it worked for you.

I'm not sure what you mean. The -nographic option doesn't help, and Suse doesn't even come far enough to switch to graphics mode. It dies in the bootloader.

Quoted 28 lines

Quoted 18 lines
OpenSolaris XXL gets hit even worse:
Quoted 2 lines
unhandled vm exit: 0x80000021
rax 0000000060000010 rbx 0000000000002000 rcx 000000000102808c rdx
0000000000101e50
Quoted 1 line
rsi 0000000000004f15 rdi 0000000000000000 rsp 0000000000004ffc rbp
0000000001028064
Quoted 1 line
r8 00000000010113c8 r9 00000000000007ff r10 00000000e0000011 r11
b7f04674b7f03ff4
Quoted 1 line
r12 bfa68a80b7f044c8 r13 b7f04674b7efae3c r14 00000001b79a03f0 r15
Quoted 8 lines
0000000000000001
Quoted 2 lines
rip 00000000000021ab rflags 00223002
Aborted
This one obviously points to an error on VM entry, according to the Intel docs. Maybe the Solaris bootloader is messing with registers it shouldn't touch?

Quoted 6 lines

Right about this caused by problem in VM entry.
99% chance Solaris is not the blame (1% they are using the BIG-real-mode).
Might be also a graphical boot loader problem, there is a debug.c file that I need to revive that tells what bothered Intel.
In general, Intel tests to deeply before vmentery.

This also happens while loading the kernel, in text mode.

Did I mention that kvm dies a horrible death ("Aiee, killing interrupt handler!") when I configure the kernel with CONFIG_PREEMPT=y? Currently I use CONFIG_PREEMPT_NONE.

Signature · 8 lines

Michael "Tired" Riepe <[email protected]>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Avi Kivity

2006-12-12 07:52 UTC #16 lore raw

In reply to a message not in the archive

Quoted 12 lines · Dor Laor

Quoted 5 lines
Windows XP Professional works, but only if I turn off ACPI (-no-acpi). The only thing that fails is poweroff/reboot (well, no ACPI, no way to turn off the power, right?) but you can do that manually. If I turn
ACPI
Quoted 4 lines
on, I get a bluescreen complaining about "IRQL_NOT_LESS_OR_EQUAL", whatever that means.

Quoted 7 lines

KVM has an issue with APIC. Currently the APIC is implemented within Qemu.
I will send a patch that moves the APIC into the kernel that fixes the problem you had above by the end of the week *so stay tuned*. There is a simple workaround detailed in the KVM faq.

The -no-apic option seems to be an even easier workaround. I'll update the faq.

It also explains why some of my guests don't have a problem -- they were installed with a pre-acpi qemu, hence they don't use the apic hal.

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Avi Kivity

2006-12-12 07:52 UTC #8 lore raw

In reply to a message not in the archive

Quoted 4 lines · Michael Riepe

Did I mention that kvm dies a horrible death ("Aiee, killing interrupt handler!") when I configure the kernel with CONFIG_PREEMPT=y? Currently I use CONFIG_PREEMPT_NONE.

I'll investigate this. I thought we had this fixed.

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Michael Riepe

2006-12-12 08:00 UTC #10 lore raw

In reply to a message not in the archive

Quoted 8 lines · Avi Kivity

Quoted 4 lines · Michael Riepe
Did I mention that kvm dies a horrible death ("Aiee, killing interrupt handler!") when I configure the kernel with CONFIG_PREEMPT=y? Currently I use CONFIG_PREEMPT_NONE.
I'll investigate this. I thought we had this fixed.

Maybe the problem was CONFIG_PREEMPT_BKL (which was set as well). Got to check that again.

Signature · 8 lines

Michael "Tired" Riepe <[email protected]>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Gerd Hoffmann

2006-12-12 09:25 UTC #13 lore raw

In reply to a message not in the archive

Hi,

Quoted 3 lines

I'm not sure what you mean. The -nographic option doesn't help, and Suse doesn't even come far enough to switch to graphics mode. It dies in the bootloader.

both isolinux and grub fail to switch into gfx mode, which likely is caused by either incomplete real mode emulation or big real mode.

Workaround for grub: just turn off the eye candy, comment the gfxmenu line in /boot/grub/menu.lst.

Kicking the installation is a bit harder as you can't just go and edit the isolinux.cfg on the cdrom. One way is to use normal qemu (without kvm acceleration) for installation and fixing up menu.lst, then go ahead with the kvm version ...

reproduce: fetch the boot iso[1], boot with "-cdrom /path/to/mini.iso -boot d"

cheers,
Gerd

[1]
http://ftp.opensuse.org/pub/opensuse/distribution/10.2/iso/cd/openSUSE-10.2-GM-i386-mini.iso

Signature · 7 lines

Gerd Hoffmann <[email protected]>

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Avi Kivity

2006-12-12 10:49 UTC #12 lore raw

In reply to a message not in the archive

Quoted 10 lines · Michael Riepe

Quoted 8 lines · Avi Kivity
Quoted 5 lines · Michael Riepe
Did I mention that kvm dies a horrible death ("Aiee, killing interrupt handler!") when I configure the kernel with CONFIG_PREEMPT=y? Currently I use CONFIG_PREEMPT_NONE.

Quoted 7 lines

Quoted 2 lines
I'll investigate this. I thought we had this fixed.
Maybe the problem was CONFIG_PREEMPT_BKL (which was set as well). Got to check that again.

If you were running from the trunk, then that would explain this. The trunk did not have the patch until a short while ago.

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Avi Kivity

Applied

2006-12-12 10:59 UTC #17 lore raw

In reply to a message not in the archive

Quoted 3 lines · Michael Riepe

The patch I attached is not essential. It fixes two error messages that occur on a 32-bit host (where the offending registers don't exist).

Applied. Please add a proper Signed-off-by: line in future patches.

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Avi Kivity

2006-12-12 11:33 UTC #18 lore raw

In reply to a message not in the archive

Quoted 3 lines · Michael Riepe

By the way: I get myriads of "kvm: unhandled rdmsr: c0000081" messages in my syslog, but they don't seem to cause any further harm.

Which guest does this happen with? AFAICT this should be handled well (vmx.c:vmx_get_msr() -> find_msr_entry())

If you'd like to debug this, you can add printk()s to see where vmx_get_msr() loses its way.

Signature · 8 lines

error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Michael Riepe

2006-12-12 16:20 UTC #15 lore raw

In reply to a message not in the archive

Hi!

Quoted 5 lines · Gerd Hoffmann

both isolinux and grub fail to switch into gfx mode, which likely is caused by either incomplete real mode emulation or big real mode.
Workaround for grub: just turn off the eye candy, comment the gfxmenu line in /boot/grub/menu.lst.

Yep, that did the trick :-)

Quoted 4 lines

Kicking the installation is a bit harder as you can't just go and edit the isolinux.cfg on the cdrom. One way is to use normal qemu (without kvm acceleration) for installation and fixing up menu.lst, then go ahead with the kvm version ...

That's how I got the system installed.

Signature · 8 lines

Michael "Tired" Riepe <[email protected]>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Michael Riepe

2006-12-12 17:42 UTC #20 lore raw

In reply to a message not in the archive

Hi!

Quoted 9 lines · Avi Kivity

Quoted 3 lines · Michael Riepe
By the way: I get myriads of "kvm: unhandled rdmsr: c0000081" messages in my syslog, but they don't seem to cause any further harm.
Which guest does this happen with? AFAICT this should be handled well (vmx.c:vmx_get_msr() -> find_msr_entry())

With all of them.

Quoted 2 lines

If you'd like to debug this, you can add printk()s to see where vmx_get_msr() loses its way.

vmx_get_msr() actually works fine. The problem is in vmx_vcpu_setup(), in the loop starting at line 1142. It tries to read all the MSRs listed in vmx_msr_index, but it fails for MSR_K6_STAR. With earlier releases, I also used to see the kernel message "kvm: msrs: 1" - that is, only MSR_EFER is found. According to Intel's docs, that's correct - the register does not exist on a Core Duo which isn't 64-bit capable (but, of course, on a Core 2 Duo - or a Xeon 51xx/53xx, aka "Woodcrest" and "Clovertown").

Signature · 8 lines

Michael "Tired" Riepe <[email protected]>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Avi Kivity

2006-12-12 18:05 UTC #22 lore raw

In reply to a message not in the archive

Quoted 9 lines · Michael Riepe

vmx_get_msr() actually works fine. The problem is in vmx_vcpu_setup(), in the loop starting at line 1142. It tries to read all the MSRs listed in vmx_msr_index, but it fails for MSR_K6_STAR. With earlier releases, I also used to see the kernel message "kvm: msrs: 1" - that is, only MSR_EFER is found. According to Intel's docs, that's correct - the register does not exist on a Core Duo which isn't 64-bit capable (but, of course, on a Core 2 Duo - or a Xeon 51xx/53xx, aka "Woodcrest" and "Clovertown").

I see now. So the fix would be to tell the cpu the syscall feature doesn't exist, presumably via cpuid (which is handled in userspace).

Signature · 8 lines

Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV