kvm-70 release

This release was cooking for far too long, and as a result there are plenty of changes. Windows XP SMP performance is much improved for AMD hosts or Intel hosts without FlexPriority (performance with FlexPriority should already be very good). There are many smp and timekeeping fixes included. mmio coalescing and async virtio-block should improve mmio performance. SLES 10 is supported as host.

Have fun reading the changelog. There _will_ be a test.

Changes from kvm-69:
- improve virtio net performance (Anthony Liguori)
- enable pci slot identification for OS X (Alexander Graf)
- fix guest reset (Jan Kiszka)
- code cleanups (Jan Kiszka)
- fix userspace compilation on older kernels (Chris Wright)
- support running on 2.6.16 hosts (Andrea Arcangeli)
- including SLES 10
- generate mptable even for uniprocessor guests
- helps esx get further along
- fix corrupted breakpoints (Jerone Young)
- on_vcpu() mechanism to execute a function on a vcpu thread
- fix 'info registers' monitor command
- fix vcpu register access livelock when a vcpu is spinning
- fix stopping on debug events (Jan Kiszka)
- fix livelocks in guest debugger (Jan Kiszka)
- add mtrr support
- helps esx limp a bit more
- fix incorrect pmtimer overflow interrupt generation
- add 'push imm' instruction to testsuite
- fix on_vcpu() during guest debug (Jan Kiszka)
- document boot option for drive flag (Carlo Marcelo Arenas Belon)
- coalesced mmio support (Laurent Vivier)
- should speed up some workloads
- allow kprobes to coexist with preempt notifier emulation (Andrea Arcangeli)
- allow virtio devices to reset themselves (Marcelo Tosatti)
- asynchronous virtio storage device (Marcelo Tosatti)
- should improve performance
- fix deadlock waiting for I/O completion
- merge qemu-svn
- remove dependency on gcc 3
- vnc client segfault fix
- fix vnc problems on resize
- qemu-nbd tool
- fix libkvm memory slot destruction api (Ben-Ami Yassour)
- enable tpr optimization for smp
- should drastically speed up Windows XP SMP guests
- big-endian kvmtrace support (Li Tan)
- remove unneeded variable (Glauber Costa)
- convert acpi tables to use the C preprocessor for less repetition
- external module compatibility for NMI (Alexander Graf)
- fix live migration with new on_vcpu() calls
- fix false flooding when a pte points to a page table
- improves 2.4 pae guests with CONFIG_HIGHPTE
- enable NMI support on Intel (Sheng Yang)
- fix zero extension during mmio on ia64 (Jes Sorensen)
- fix hypercall return value on AMD
- fixes running CONFIG_KVM_GUEST guests
- s390: use yield() instead of schedule() (Christian Borntraeger)
- s390: fix locking order problem (Christian Borntraeger)
- s390: handle machine checks while guest is running (Christian Borntraeger)
- s390: fix interrupt delivery (Carsen Otte)
- s390: send program check on access error (Carsen Otte)
- s390: fix race when idling (Carsen Otte)
- ppc: add lwzx/stwz instruction emulation (Hollis Blanchard)
- ppc: code cleanup (Hollis Blanchard)
- ppc: report bad gfns (Hollis Blanchard)
- optimize pio path slightly
- improve segment register ordering
- migrate pit timer on vcpu migration (Marcelo Tosatti)
- emulate jmp far instruction (Guillaume Thouvenin)
- emulate mov reg, imm instruction (Guillaume Thouvenin)
- emulate mov reg, sreg instruction (Guillaume Thouvenin)
- optimize mmu page prefetching
- emulate push imm instruction
- avoid mmu page prefetching on AMD
- allow VMware Server to take over vmx when unloading the module (Eli Collins)
- avoid spinning during large guest teardown
- fix ioapic interrupt sharing (Marcelo Tosatti)
- fix crash on resume from suspend to ram (Andrea Arcangeli)
- fix race between timer migration and vcpu migration (Marcelo Tosatti)
- close timer injection race when entering guest mode (Marcelo Tosatti)
- fix reverse map corruption with large pages (Marcelo Tosatti)
- fix large page pte updates on non-pae guests (Marcelo Tosatti)
- avoid timer injection if timer already pending (Marcelo Tosatti)
- avoid linear rip calculation if not reporting (Glauber Costa)
- fix host oops on guest userspace access to guest page table
- improve emulation failure reporting
- emulate nop and xchg reg, acc (Mohammed Gamal)
- handle undecoded rex.b when r/m = 5
- simplify r/m and sib decoding
- simplify relative rip decoding
- avoid segment adjust for lea instruction

Notes:
If you use the modules bundled with kvm-70, you can use any version
of Linux from 2.6.17 upwards. You may also use kvm-70 userspace with the kvm modules provided by Linux 2.6.22 or above. Some features may only be available in newer releases.

http://kvm.qumranet.com

Excellent news !

Quick question: when will you merge the recent qemu-img.c changes (qemu svn rev 4672) ?

-marc

I'll try to do this soon.

I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

on our centos-5 host we run our guests (as you suggested) with clocksource=acpi_pm does it still required with this release?

  Levente                               "Si vis pacem para bellum!"

I'd stick with acpi_pm, especially on AMD (which is what you're running on IIRC).

I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

kvm-70 do not compile on centos-5 kernel-2.6.18-53.1.21.el5, there are 2 warning (which would be nice to fix anyway), but the real problems is here:
--------------------------------------------------------------
WARNING: /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm-intel.o - Section mismatch: reference to .init.text: from .text.fixup after '' (at offset 0x97)
WARNING: "kallsyms_lookup_name" [/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm.ko] undefined!
--------------------------------------------------------------
this's the full buildlog:
--------------------------------------------------------------
make KDIR=/usr/src/kernels/2.6.18-53.1.21.el5-x86_64

make -C /usr/src/kernels/2.6.18-53.1.21.el5-x86_64 M=`pwd` \
		LINUXINCLUDE="-I`pwd`/include -Iinclude -I`pwd`/include-compat \
		-include include/linux/autoconf.h \
		-include `pwd`/external-module-compat.h"

make[1]: Entering directory `/usr/src/kernels/2.6.18-53.1.21.el5-x86_64'
LD /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/built-in.o
CC [M] /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/svm.o CC [M]
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/external-module-compat.o

  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/vmx.o
  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/vmx-debug.o
  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm_main.o
  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/x86.o
  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/mmu.o

CC [M]
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/x86_emulate.o
CC [M]
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/anon_inodes.o

  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/irq.o
  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/i8259.o
  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/lapic.o
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/lapic.c: In function 'kvm_create_lapic':
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/lapic.c:1020: warning: assignment from incompatible pointer type
  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/ioapic.o
  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/preempt.o
  CC [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/i8254.o

/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/i8254.c: In function 'create_pit_timer':
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/i8254.c:269: warning: assignment from incompatible pointer type
CC [M]
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/coalesced_mmio.o

  LD [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm.o
  LD [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm-intel.o
  LD [M]  /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm-amd.o

Building modules, stage 2.
MODPOST
WARNING: /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm-intel.o - Section mismatch: reference to .init.text: from .text.fixup after '' (at offset 0x97)
WARNING: "kallsyms_lookup_name" [/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm.ko] undefined!
CC
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm-amd.mod.o
LD [M] /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm-amd.ko
CC
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm-intel.mod.o
LD [M]
/home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm-intel.ko
CC /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm.mod.o
LD [M] /home/robot/rpm/BUILD/kvm-kmod-70/_kmod_build_/kernel/kvm.ko
make[1]: Leaving directory `/usr/src/kernels/2.6.18-53.1.21.el5-x86_64'
--------------------------------------------------------------
and of course if i try to install this module, then i got:
--------------------------------------------------------------
kvm: Unknown symbol kallsyms_lookup_name
kvm_intel: Unknown symbol kvm_clear_guest_page
kvm_intel: Unknown symbol kvm_exit
kvm_intel: Unknown symbol kvm_init
kvm_intel: Unknown symbol kvm_enable_efer_bits
kvm_intel: Unknown symbol kvm_timer_intr_post
kvm_intel: Unknown symbol kvm_mmu_set_nonpresent_ptes
kvm_intel: Unknown symbol gfn_to_page
kvm_intel: Unknown symbol segment_base
kvm_intel: Unknown symbol kvm_get_msr_common
kvm_intel: Unknown symbol __kvm_set_memory_region
kvm_intel: Unknown symbol kvm_vcpu_uninit
kvm_intel: Unknown symbol kvm_emulate_halt
kvm_intel: Unknown symbol kvm_set_apic_base
kvm_intel: Unknown symbol kvm_lapic_find_highest_irr
kvm_intel: Unknown symbol kvm_task_switch
kvm_intel: Unknown symbol kvm_enable_tdp
kvm_intel: Unknown symbol kvm_lmsw
kvm_intel: Unknown symbol kvm_set_memory_region
kvm_intel: Unknown symbol kvm_queue_exception
kvm_intel: Unknown symbol emulate_instruction
kvm_intel: Unknown symbol kvm_write_guest_page
kvm_intel: Unknown symbol fx_init
kvm_intel: Unknown symbol kvm_cpu_has_interrupt
kvm_intel: Unknown symbol kvm_lapic_get_cr8
kvm_intel: Unknown symbol kvm_set_cr3
kvm_intel: Unknown symbol kvm_get_cr8
kvm_intel: Unknown symbol kvm_vcpu_cache
kvm_intel: Unknown symbol kvm_emulate_hypercall
kvm_intel: Unknown symbol load_pdptrs
kvm_intel: Unknown symbol kvm_handle_fault_on_reboot
kvm_intel: Unknown symbol kvm_set_cr4
kvm_intel: Unknown symbol kvm_set_cr0
kvm_intel: Unknown symbol kvm_set_cr8
kvm_intel: Unknown symbol kvm_lapic_enabled
kvm_intel: Unknown symbol kvm_mmu_page_fault
kvm_intel: Unknown symbol kvm_mmu_reset_context
kvm_intel: Unknown symbol kvm_queue_exception_e
kvm_intel: Unknown symbol kvm_emulate_cpuid
kvm_intel: Unknown symbol kvm_vcpu_init
kvm_intel: Unknown symbol gfn_to_hva
kvm_intel: Unknown symbol kvm_set_msr_common
kvm_intel: Unknown symbol kvm_mmu_set_base_ptes
kvm_intel: Unknown symbol kvm_cpu_get_interrupt
kvm_intel: Unknown symbol kvm_emulate_pio
kvm_intel: Unknown symbol kvm_mmu_set_mask_ptes
kvm_intel: Unknown symbol kvm_is_error_hva
--------------------------------------------------------------

  Levente                               "Si vis pacem para bellum!"

Easy enough to work around, as kallsyms_lookup_name is only used in one place:

diff -ru kvm-70/kernel/external-module-compat.h kvm-70.new/kernel/external-module-compat.h
@@ -663,7 +663,7 @@
                              const enum hrtimer_mode mode);
 extern int (*hrtimer_cancel_p)(struct hrtimer *timer);
 
-#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19) && defined(CONFIG_KALLSYMS)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19) && defined(CONFIG_KALLSYMS) && 0
 static inline void hrtimer_kallsyms_resolve(void)
 {
        hrtimer_init_p = (void *) kallsyms_lookup_name("hrtimer_init");

Something's weird, if CONFIG_KALLSYMS isn't defined, the above code shouldn't be selected already. Perhaps I'm missing something.

Farkas, can you send me /proc/config.gz by PM thanks!

CONFIG_KALLSYMS is indeed defined, but kallsyms_lookup_name doesn't appear to be exported in this kernel.

Ok I'm aware that kallsyms_lookup_name isn't exported in recent kernels, and when I implemented the patch I verified that the export was removed here:

changeset:   38128:da689f9ecdd9
user:        Ananth N Mavinakayanahalli <[email protected]>
date:        Mon Oct 02 21:57:16 2006 +0700
summary:     [PATCH] Kprobes: Make kprobe modules more portable

diff -r 025d3a916c7d -r da689f9ecdd9 kernel/kallsyms.c
@@ -154,7 +154,6 @@ unsigned long kallsyms_lookup_name(const
        }
        return module_kallsyms_lookup_name(name);
 }
-EXPORT_SYMBOL_GPL(kallsyms_lookup_name);
 
 

v2.6.19-rc3                    40489:bed003f043c8
v2.6.19-rc2                    39833:93988811847a
v2.6.19-rc1                    39283:faecb9cc26cd
v2.6.18                        34283:dc1d277d06e0

That's where the 2.6.19 number comes from. So 2.6.18 should still have it because it has changeset number 34283.

So I guess RHEL was patched with v2.6.19-rc1 while keeping the name to 2.6.18.

hrtimers were exported here:

changeset:   27310:a671ca535b55
user:        Stephen Hemminger <[email protected]>
date:        Thu Jun 01 06:27:11 2006 +0700
summary:     [PATCH] hrtimer: export symbols

diff -r 85c4a996a332 -r a671ca535b55 kernel/hrtimer.c
@@ -456,6 +456,7 @@ hrtimer_start(struct hrtimer *timer, kti
 
        return ret;
 }
+EXPORT_SYMBOL_GPL(hrtimer_start);
 
 /**
  * hrtimer_try_to_cancel - try to deactivate a timer

[..]

v2.6.17                        27472:e3554576c29a
v2.6.17-rc6                    27401:d349717b8dc8
v2.6.17-rc5                    27216:016512c08f6b
v2.6.17-rc4                    26901:c86d3036bd16
v2.6.17-rc3                    26461:8a0c5ce16e31
v2.6.17-rc2                    26210:cbfa59dad07d
v2.6.17-rc1                    25555:969a12bcc0a9
v2.6.16                        21346:7d239c83edea

So the hrtimer exports are available in 2.6.17 included. I was going strict on the kallsyms_lookup removal point, but it is effectively safer to go strict on the hrtimer export because distros will keep their version number backwards instead of forward.

So to fix this we've only to apply this:

index 08b8841..79b8373 100644
@@ -663,7 +663,7 @@ extern int (*hrtimer_start_p)(struct hrtimer *timer, ktime_t tim,
 			      const enum hrtimer_mode mode);
 extern int (*hrtimer_cancel_p)(struct hrtimer *timer);
 
-#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,19) && defined(CONFIG_KALLSYMS)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,17) && defined(CONFIG_KALLSYMS)
 static inline void hrtimer_kallsyms_resolve(void)
 {
 	hrtimer_init_p = (void *) kallsyms_lookup_name("hrtimer_init");

fyi this the standard latest rhel/centos-5.1 kernel, so if you want to run kvm on these distros it'd be nice to fix:-)

  Levente                               "Si vis pacem para bellum!"

do you mean /boot/config*?

  Levente                               "Si vis pacem para bellum!"

This one's harmless (though I'd like to remove it).

This is now fixed in kvm-userspace.git.

I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.