Skip to content

RFC VMI i386 Linux virtualization interface proposal

Zachary Amsden lkml xen-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest24 patches 136 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/24i386 Vmi documentationDocumentation/vmi_spec.txt+2197 -023 replies
1/24i386 Vmi documentationDocumentation/vmi_spec.txt+2197 -00 replies
2/24i386 Vmi configarch/i386/Kconfig arch/i386/Kconfig.cpu+41 -76 replies
2/24i386 Vmi configarch/i386/Kconfig arch/i386/Kconfig.cpu+41 -70 replies
3/24i386 Vmi interface definitioninclude/asm-i386/mach-vmi/paravirtualInterface.h include/asm-i386/mach-vmi/vmiCalls.h+286 -05 replies
3/24i386 Vmi interface definitioninclude/asm-i386/mach-vmi/paravirtualInterface.h include/asm-i386/mach-vmi/vmiCalls.h+286 -00 replies
4/24i386 Vmi inline implementationarch/i386/Kconfig include/asm-i386/mach-vmi/mach_asm.h include/asm-i386/mach-vmi/vmi.h+481 -04 replies
4/24i386 Vmi inline implementationarch/i386/Kconfig include/asm-i386/mach-vmi/mach_asm.h include/asm-i386/mach-vmi/vmi.h+481 -00 replies
5/24i386 Vmi code patchingarch/i386/Makefile arch/i386/mach-vmi/Makefile arch/i386/mach-vmi/setup.c +4 more+470 -032 replies
6/24i386 Vmi magic fixesarch/i386/kernel/vmlinux.lds.S include/asm-i386/mach-default/mach-vmlinux.lds.h include/asm-i386/mach-vmi/mach-vmlinux.lds.h +1 more+60 -10 replies
7/24i386 Vmi memory holearch/i386/Kconfig include/asm-i386/fixmap.h+10 -112 replies
8/24i386 Vmi syscall assemblyarch/i386/kernel/entry.S include/asm-i386/mach-default/mach_asm.h include/asm-i386/spinlock.h+37 -202 replies
9/24i386 Vmi smp supportarch/i386/kernel/smpboot.c arch/i386/mach-vmi/Makefile arch/i386/mach-vmi/smpboot_hooks.c +3 more+239 -24 replies
10/24i386 Vmi descriptor changesarch/i386/kernel/traps.c include/asm-i386/desc.h include/asm-i386/mach-default/mach_desc.h +1 more+260 -664 replies
11/24i386 Vmi segment changesarch/i386/kernel/entry.S arch/i386/kernel/process.c arch/i386/kernel/traps.c +4 more+118 -200 replies
12/24i386 Vmi processor headerinclude/asm-i386/mach-default/mach_processor.h include/asm-i386/mach-vmi/mach_processor.h include/asm-i386/processor.h+252 -1000 replies
13/24i386 Vmi system headerinclude/asm-i386/mach-default/mach_system.h include/asm-i386/mach-vmi/mach_system.h include/asm-i386/system.h+299 -740 replies
14/24i386 Vmi reboot fixesarch/i386/kernel/crash.c arch/i386/kernel/process.c arch/i386/kernel/reboot.c +2 more+22 -124 replies
15/24i386 Vmi apic headerinclude/asm-i386/apic.h include/asm-i386/mach-default/mach_apicops.h include/asm-i386/mach-vmi/mach_apicops.h+93 -190 replies
16/24i386 Vmi io headerinclude/asm-i386/io.h include/asm-i386/mach-default/mach_io.h include/asm-i386/mach-vmi/mach_io.h+184 -142 replies
17/24i386 Vmi msr patchinclude/asm-i386/mach-default/mach_msr.h include/asm-i386/mach-vmi/mach_msr.h include/asm-i386/msr.h+111 -265 replies
18/24i386 Vmi tlbflush headerinclude/asm-i386/mach-default/mach_tlbflush.h include/asm-i386/mach-vmi/mach_tlbflush.h include/asm-i386/tlbflush.h+100 -350 replies
19/24i386 Vmi mmu changesarch/i386/kernel/efi_stub.S arch/i386/kernel/head.S arch/i386/kernel/process.c +17 more+485 -1051 reply
20/24i386 Vmi module fixupsinclude/asm-i386/mach-default/mach_module.h include/asm-i386/mach-vmi/mach_module.h include/asm-i386/module.h+16 -10 replies
21/24i386 Vmi proc nodearch/i386/mach-vmi/Makefile arch/i386/mach-vmi/proc.c+195 -11 reply
22/24i386 Consolidate redundant timer codearch/i386/kernel/apic.c arch/i386/kernel/i8259.c arch/i386/kernel/io_apic.c +10 more+136 -880 replies
23/24i386 Vmi timer patcharch/i386/Kconfig arch/i386/kernel/timers/Makefile arch/i386/kernel/timers/timer.c +11 more+771 -130 replies
24/24i386 Vmi no idle hzarch/i386/Kconfig arch/i386/kernel/apic.c arch/i386/kernel/irq.c +9 more+298 -182 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC, PATCH 0/24] VMI i386 Linux virtualization interface proposal

In OLS 2005, we described the work that we have been doing in VMware with respect a common interface for paravirtualization of Linux. We shared the general vision in Rik's virtualization BoF.

This note is an update on our further work on the Virtual Machine Interface, VMI. The patches provided have been tested on 2.6.16-rc6. We are currently recollecting performance information for the new -rc6 kernel, but expect our numbers to match previous results, which showed no impact whatsoever on macro benchmarks, and nearly neglible impact on microbenchmarks.

Unlike the full-virtualization techniques used in the traditional VMware products, paravirtualization is a technique where the operating system is modified to enlighten the hypervisor with timely knowledge about the operating system's activities. Since the hypervisor now depends on the kernel to tell it about common idioms etc, it does not need to write protect OS objects such as page and descriptor tables as a solution based on full-virtualization needs. This has two important effects (a) it shortens the critical path, since faulting is expensive on modern processors (b) by eliminating complex heuristics the hypervisor is simplified. While the former delivers performance, the latter is quite important too.

Not surprisingly, paravirtualization's strength, ie that it encourages tighter communication between the kernel and the hypervisor, is also its weakness. Unless the changes to the operating system are moderated, you can very quickly find yourself with a kernel that (a) looks and feels like a brand new kernel or (b) cannot run on native machines or on newer versions of the hypervisor without a full recompile. The former can impede innovation in the Linux kernel, and the latter can be a problem for software vendors.

VMware proposes VMI as a paravirtualization interface for Linux that solves these problems.
- A VMI'fied Linux kernel runs unmodified on native hardware, and on many hypervisors, while simultaneously delivering on the performance promise of paravirtualization.
- VMI has a rich and low level interface, which allows the kernel to cope with future hardware evolution by querying for hardware capability. It is our expectation that a single kernel will run unmodified on both today's processors with limited hardware virtualization support and also keep up with any evolution on the processor front
- VMI Linux is a fairly clean interface, with distinct name spaces for objects from the kernel and the hypervisor. Nowhere do we mingle names from the hypervisor with that of the kernel. This separation allows innovation in the kernel to proceed at the same speed as always. For most kernel developers, a VMI kernel looks and feels like a regular Linux kernel.
- VMI Linux still supports "native" hypervisor device drivers, for example a hypervisor vendor's own private network or block device drivers which are free to use any interface desired to communicate with the hypervisor.

At present, we are sharing a working implementation of the VMI for 2.6.16-rc6 version of Linux. We have verified that VMI Linux does indeed run well on native machines (both P4 and Opterons), and on VMware style hypervisors. VMI Linux has negligible overheads on native machines, so much so, that we are confident that VMI Linux can, in the long run, be the default Linux for i386. We believe that this interface is both cleaner and more powerful than other proposals that have been made towards virtualization of Linux, and can easily be adapted to work with other hypervisors.

This is by no means finished work. A few of the areas that need more attention and exploration are (a) 64bit support is still lacking, but we feel a port of VMI to the 64 bit Linux can be done without too much trouble (b) the Xen compatibility layer needs some work to bring it up to the Xen 3.0 interfaces. Work is underway on this already, and no major issues are expected at this time.

Two final notes. This is not an attempt to force a proprietary interface into the Linux kernel. This is an attempt to find a common interface that can be used by many hypervisors by isolating hypervisor specific idioms into a neutral layer. This new layer is just what is claims to be - a virtual machine interface, which allows hypervisor dependent code to be abstracted in a way that benefits both Linux and hypervisor development.

This is also not an attempt to define an exact and final specification of how virtualization should be done in Linux. This is very much a work in progress, and it is understood that the interfaces proposed here will change in time to accommodate the needs of all interested parties. We hope to find a common solution that can eventually become part of the Linux kernel and serve as a model for other operating systems as well.

We appreciate your feedback on this design and the patches to Linux, and welcome working with anyone who is interested in making virtualization in Linux a friendly environment to innovate in. If you find the ideas here interesting, please volunteer to help improve them.