Skip to contentSkip to conversation

i386 Vmi inline implementation

RFC PATCH In review Zachary Amsden 3 participants 5 messages Mar 13, 2006 – Mar 22, 2006

lore
Share & tools
lore
#0 lore raw

RFC PATCH 4/24 i386 Vmi inline implementation

Macros to use VMI calls from assembly and C languages are introduced. The macros are quite complex, but the end result is rather impressive. The result is that when compiling a VMI kernel, the native code is emitted inline, with no function call overhead, and some wiggle room for register allocation. The hypervisor compatibility code is emitted out of line into a separate section, and patched dynamically during the process of preparing the kernel for boot. In the end, this task is envisioned as being done by a loader agent outside of the kernel proper, allowing for a completely transparent kernel start of day.

Please forgive some of the historical and clearly in-transition definitions in vmi.h; in particular, the MMU VMI calls have not yet been adapted to the inline interface, and the naming convention has not yet been Linux-ified. Also, there are artifacts remaining here from the compile time ability to choose a register convention. There are also some bugs with non-regparm compilation.

The set of calls in the VMI interface here needs some minor adjustments for Xen compatibility. In particular, a method to convert machine to physical mappings in PTEs is one missing interface, and with that and some other rather small changes, a VMI kernel is perfectly capable of running in a Xen hypervisor.

Many thanks to Joshua LeVasseur at the University of Karlsruhe, and Volkmar Uhlig, now at IBM, who verified this claim on a slightly different form of the interface. We are actively working on assembling all of those bits together right now to work on the Xen 3.0 interface. The end result is that the exact same kernel binary can run on native hardware as well as multiple hypervisors with near zero overhead, and absolutely minimal impact of the hypervisor based idioms on the native code base.

Note that hypervisor specific drivers need not ever be a part of the core interface, as drivers are already very well encapsulated by kernel modules, and are free to use any interface they wish to talk to the hardware (in this case, the hypervisor). Imposing an interface such as this on the driver is somewhat non-productive, and needlessly complicates both the driver and the core interface. There is a little bit of blending room for devices like timers, interrupt controllers, and other components that are closely coupled with the CPU, but with good modularization in the kernel, these devices can largely be factored out as well.

Forgive the strangeness introduced by compile time options for controlling the calling convention; currently, non-REGPARM builds are actually broken. I plan on fixing this so that smart inliners can take advantage of known constants; these constants are available in the decoded call stream as push immediate instructions, but they are not generated for the first three arguments if the regparm calling convention is used, since the register allocation causes that data to go out of our scope. The exact call ABI is still up for debate, so this is supposed to allow some tinkering for CPI and code overhead measurement. Eventually, one ABI will be chosen, and the macros can get a bit simpler.

include/asm-i386/mach-vmi/mach_asm.h +191 -0
Index: linux-2.6.16-rc5/include/asm-i386/mach-vmi/mach_asm.h
===================================================================
@@ -0,0 +1,191 @@
+#ifndef __MACH_ASM_H
+#define __MACH_ASM_H
+
+/*
+ * Take care to ensure that these labels do not overlap any in parent scope,
+ * as the breakage can be difficult to detect.  I haven't found a flawless
+ * way to avoid collision because local \@ can not be saved for a future
+ * macro definitions, and we are stuck running cpp -traditional on entry.S 
+ *
+ * To work around gas bugs, we must emit the native sequence here into a
+ * separate section first to measure the length.  Some versions of gas have
+ * difficulty resolving vmi_native_end - vmi_native_begin during evaluation
+ * of an assembler conditional, which we use during the .rept directive
+ * below to generate the nop padding -- Zach
+ */
+
+/* First, measure the native instruction sequence length */
+#define vmi_native_start			\
+	.pushsection .vmi.native,"ax";		\
+	771:;
+#define vmi_native_finish			\
+	772:;					\
+	.popsection;
+#define vmi_native_begin	771b
+#define vmi_native_end		772b
+#define vmi_native_len		(vmi_native_end - vmi_native_begin)
+
+/* Now, measure and emit the vmi translation sequence */
+#define vmi_translation_start			\
+	.pushsection .vmi.translation,"ax";	\
+	781:;
+#define vmi_translation_finish			\
+	782:;					\
+	.popsection;
+#define vmi_translation_begin	781b
+#define vmi_translation_end	782b
+#define vmi_translation_len	(vmi_translation_end - vmi_translation_begin)
+
+/* Finally, emit the padded native sequence */
+#define vmi_padded_start				\
+	791:;
+#define vmi_padded_finish				\
+	792:;
+#define vmi_padded_begin	791b
+#define vmi_padded_end		792b
+#define vmi_padded_len		(vmi_padded_end - vmi_padded_begin)
+
+#define vmi_call(name)						\
+	call .+5+name
+
+/*
+ * Pad out the current native instruction sequence with a series of nops;
+ * the nop used is 0x66* 0x90 which is data16 nop or xchg %ax, %ax.  We
+ * pad out with up to 11 bytes of prefix at a time because beyond that
+ * both AMD and Intel processors start to show inefficiencies.
+ */
+#define vmi_nop_pad						\
+.equ vmi_pad_total, vmi_translation_len - vmi_native_len;	\
+.equ vmi_pad, vmi_pad_total;					\
+.rept (vmi_pad+11)/12;						\
+	.if vmi_pad > 12;					\
+		.equ vmi_cur_pad, 12;				\
+	.else;							\
+		.equ vmi_cur_pad, vmi_pad;			\
+	.endif;							\
+	.if vmi_cur_pad > 1;					\
+		.fill vmi_cur_pad-1, 1, 0x66;			\
+	.endif;							\
+	.byte 0x90;						\
+	.equ vmi_pad, vmi_pad - vmi_cur_pad;			\
+.endr;
+
+/*
+ * Create an annotation for a VMI call; the VMI call currently must be
+ * wrapped in one of the vmi_raw_call (for assembler) or one of the
+ * family of defined wrappers for C code.
+ * XXXPara - use local labels
+ */
+#define vmi_annotate(name)				\
+	.pushsection .vmi.annotation,"a";		\
+	.align 4;					\
+	.long name;					\
+	.long vmi_padded_begin;				\
+	.long vmi_translation_begin;			\
+	.byte vmi_padded_len;				\
+	.byte vmi_translation_len;			\
+	.byte vmi_pad_total;				\
+	.byte 0;					\
+	.popsection;
+
+#define vmi_raw_call(name, native)			\
+	vmi_native_start;				\
+	native;						\
+	vmi_native_finish;				\
+							\
+	vmi_translation_start;				\
+	vmi_call(name);					\
+	vmi_translation_finish;				\
+							\
+	vmi_padded_start;				\
+	native;						\
+	vmi_nop_pad;					\
+	vmi_padded_finish;				\
+							\
+	vmi_annotate(name);
+
+#include <vmiCalls.h>
+#ifdef __ASSEMBLY__
+/*
+ * Create VMI_CALL_FuncName definitions for assembly code using
+ * equates; the C enumerations can not be used without propagating
+ * them in some fashion, and rather the obfuscate asm-offsets.c, it
+ * seems reasonable to confine this here.
+ */
+.equ VMI_CALL_CUR, 0;
+#define VDEF(call)				\
+	.equ VMI_CALL_/**/call, VMI_CALL_CUR;	\
+	.equ VMI_CALL_CUR, VMI_CALL_CUR+1;
+VMI_CALLS
+#undef VDEF
+#endif /* __ASSEMBLY__ */
+
+#define IRET		vmi_raw_call(VMI_CALL_IRET,	iret)
+#define CLI		vmi_raw_call(VMI_CALL_DisableInterrupts, cli)
+#define STI		vmi_raw_call(VMI_CALL_EnableInterrupts,	sti)
+#define STI_SYSEXIT	vmi_raw_call(VMI_CALL_SYSEXIT,	sti; sysexit)
+
+/*
+ * Due to the presence of "," in the instruction, and the use of
+ * -traditional to compile entry.S, we can not use a macro to
+ * encapsulate (mov %cr0, %eax); the full expansion must be
+ * written.
+ */
+#define GET_CR0		vmi_native_start;		\
+			mov %cr0, %eax;			\
+			vmi_native_finish;		\
+			vmi_translation_start;		\
+			vmi_call(VMI_CALL_GetCR0);	\
+			vmi_translation_finish;		\
+			vmi_padded_start;		\
+			mov %cr0, %eax;			\
+			vmi_nop_pad;			\
+			vmi_padded_finish;		\
+			vmi_annotate(VMI_CALL_GetCR0);
+
+#ifndef __ASSEMBLY__
+/*
+ * Several handy macro definitions used to convert the raw assembler
+ * definitions here into quoted strings for use in inline assembler
+ * from C code.
+ *
+ * To convert the value of a defined token to a string, XSTR(x)
+ * To concatenate multiple parameters separated by commas, XCONC()
+ * To convert the value of a defined value with commas, XCSTR()
+ *
+ * These macros are incompatible with -traditional
+ */
+#define MAKESTR(x)              #x
+#define XSTR(x)                 MAKESTR(x)
+#define XCONC(args...)		args
+#define CONCSTR(x...)		#x
+#define XCSTR(x...)		CONCSTR(x)
+
+/*
+ * Propagate these definitions as strings up to C code for convenient use
+ * in stringized assembler as pseudo-mnemonics; we must emit assembler
+ * directives to generate equates for the VMI_CALL_XXX symbols, since they
+ * will not be available otherwise to the assembler, and we can't emit
+ * the C versions of these functions from within an inline assembler
+ * string.
+ */
+asm(".equ VMI_CALL_CUR, 0;\n\t");
+#define VDEF(call)						\
+	asm (".equ VMI_CALL_" #call ", VMI_CALL_CUR;\n\t");	\
+	asm (".equ VMI_CALL_CUR, VMI_CALL_CUR+1;\n\t");
+VMI_CALLS
+#undef VDEF
+
+/*
+ * Sti and Cli are special cases, used in raw inline assembler strings;
+ * they do not need much of the machinery provided by C-code to do type
+ * checking or push arguments onto the stack, which means we can simply
+ * quote the assembler versions defined above rather than try to pry
+ * apart the call sites which use these raw strings.
+ */
+#define CLI_STRING	XCSTR(CLI)
+#define STI_STRING	XCSTR(STI)
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __MACH_ASM_H */
include/asm-i386/mach-vmi/vmi.h +249 -0

249 lines not on this page · raw message

arch/i386/Kconfig +41 -0

47 lines not on this page · raw message

#1 lore raw
Quoted 9 lines · Zachary Amsden

Macros to use VMI calls from assembly and C languages are introduced. The macros are quite complex, but the end result is rather impressive. The result is that when compiling a VMI kernel, the native code is emitted inline, with no function call overhead, and some wiggle room for register allocation. The hypervisor compatibility code is emitted out of line into a separate section, and patched dynamically during the process of preparing the kernel for boot. In the end, this task is envisioned as being done by a loader agent outside of the kernel proper, allowing for a completely transparent kernel start of day.

We already do runtime patching for SMP vs. UP, could you use same infrastructure? I do not want VMI-specific grub.
Pavel

Signature · 1 line
42:             i < SampleTable.Length;
#2 lore raw
Quoted 2 lines · Pavel Machek

We already do runtime patching for SMP vs. UP, could you use same infrastructure? I do not want VMI-specific grub.

I think we could almost use the same infrastructure - or extend it to work. The problem is how to determine register liveness for calls into the VMI layer which take > 3 arguments. If you squeeze all the arguments into fixed registers, you can unnecessarily constrain the native code - and rapidly run out of registers for the compiler to use, generating impossible constraints in some cases.

To work around these issues, we do not constrain the arguments beyond the first three registers. But this means that the hypervisor needs a way to locate the additional arguments. That information gets encoded implicitly by the auto-generated translation into a VMI call, which pushes the arguments onto the stack (thus revealing the registers, and allowing constant immediate optimization).

Now, if you can generate all of the code to do the callout to the VMI, you could use it directly in an alternative instruction sequence, just as the existing infrastructure. But I'm not sure you can do it in one compiler pass. We are already pushing the limits of the preprocessor, compiler and assembler here. Having another preprocessing pass may make it possible, but it does make the build more complicated.

I'm not sure it is really what we want though. The alternative instruction interfaces is based on global feature detection that is done and applied by the _kernel_. In the end, we want the hypervisor to be able to toggle each VMI call site as a separate feature, and replace it with hypervisor specific code. In this way, a VT based hypervisor could simply not patch those class of VMI calls that are already emulated by hardware, and also inline direct hypercalls for those classes that are not. The VMI layer is supposed to be very much an inline linker for feature detection done by the _hypervisor_.

Zach

#3 lore raw
In reply to Zachary Amsden

include/asm-i386/mach-vmi/mach_asm.h line 162

Quoted 5 lines · Zachary Amsden
include/asm-i386/mach-vmi/mach_asm.h +5 -0
+#define MAKESTR(x)              #x
+#define XSTR(x)                 MAKESTR(x)
+#define XCONC(args...)		args
+#define CONCSTR(x...)		#x
+#define XCSTR(x...)		CONCSTR(x)

We have legions of these all over the tree. How about you put them into some central file and gc a few?

include/asm-i386/mach-vmi/mach_asm.h line 172

Quoted 9 lines
include/asm-i386/mach-vmi/mach_asm.h +9 -0
+/*
+ * Propagate these definitions as strings up to C code for convenient use
+ * in stringized assembler as pseudo-mnemonics; we must emit assembler
+ * directives to generate equates for the VMI_CALL_XXX symbols, since they
+ * will not be available otherwise to the assembler, and we can't emit
+ * the C versions of these functions from within an inline assembler
+ * string.
+ */
+asm(".equ VMI_CALL_CUR, 0;\n\t");

The standard way to do this is to use asm-offsets.c

include/asm-i386/mach-vmi/mach_asm.h line 176

Quoted 4 lines
include/asm-i386/mach-vmi/mach_asm.h +4 -0
+#define VDEF(call)						\
+	asm (".equ VMI_CALL_" #call ", VMI_CALL_CUR;\n\t");	\
+	asm (".equ VMI_CALL_CUR, VMI_CALL_CUR+1;\n\t");
+VMI_CALLS

Hmmm? This doesn't look like something a header file should be doing.

How about you put that big list and the definition into a .c ?

Quoted 1 line

+#if defined(CONFIG_VMI_C_CONVENTION)

I don't think that file can be reviewed in any meaningful way before you don't get rid of the macro mess and the unneeded calling conventions.

-Andi

#4 lore raw
Quoted 13 lines · Andi Kleen

include/asm-i386/mach-vmi/mach_asm.h line 162

Quoted 6 lines · Zachary Amsden
include/asm-i386/mach-vmi/mach_asm.h +5 -0
+#define MAKESTR(x)              #x
+#define XSTR(x)                 MAKESTR(x)
+#define XCONC(args...)		args
+#define CONCSTR(x...)		#x
+#define XCSTR(x...)		CONCSTR(x)

We have legions of these all over the tree. How about you put them into some central file and gc a few?

Will do. I think the XCSTR() is a new one though.. I've never seen it before, so I had to invent it.

Quoted 15 lines

include/asm-i386/mach-vmi/mach_asm.h line 172

Quoted 12 lines
include/asm-i386/mach-vmi/mach_asm.h +9 -0
+/*
+ * Propagate these definitions as strings up to C code for convenient use
+ * in stringized assembler as pseudo-mnemonics; we must emit assembler
+ * directives to generate equates for the VMI_CALL_XXX symbols, since they
+ * will not be available otherwise to the assembler, and we can't emit
+ * the C versions of these functions from within an inline assembler
+ * string.
+ */
+asm(".equ VMI_CALL_CUR, 0;\n\t");

The standard way to do this is to use asm-offsets.c

Almost. This is a real shady thing to do, but this is used by C-code, not assembly, to propagate the constants for VMI_STI and VMI_CLI into the spinlock macros where they are referenced by inline assembly. We just need defined constants, but I didn't want to do that yet as we are still changing the MMU portion of the interface to map better onto Xen 3.0.

Quoted 11 lines

include/asm-i386/mach-vmi/mach_asm.h line 176

Quoted 8 lines
include/asm-i386/mach-vmi/mach_asm.h +4 -0
+#define VDEF(call)						\
+	asm (".equ VMI_CALL_" #call ", VMI_CALL_CUR;\n\t");	\
+	asm (".equ VMI_CALL_CUR, VMI_CALL_CUR+1;\n\t");
+VMI_CALLS

Hmmm? This doesn't look like something a header file should be doing.

Quoted 2 lines

How about you put that big list and the definition into a .c ?

See note above. Shady, I agree.

Quoted 7 lines
Quoted 3 lines

+#if defined(CONFIG_VMI_C_CONVENTION)

I don't think that file can be reviewed in any meaningful way before you don't get rid of the macro mess and the unneeded calling conventions.

I sort of expected that. It is a mess. You should have seen it two months ago. It was the most hideous gobbledygook I've ever written, but in the great tradition of hacking, it did the job. The results are impressive, however. Here is some disassembly of arch/i386/kernel/process.c

Disassembly of section .vmi.translation:

00000000 <.vmi.translation>:

  0:   e8 23 00 00 00          call   28 <.vmi.translation+0x28>
  5:   e8 24 00 00 00          call   2e <.vmi.translation+0x2e>
  a:   e8 23 00 00 00          call   32 <.vmi.translation+0x32>
  f:   e8 2a 00 00 00          call   3e <.vmi.translation+0x3e>

Here, you can see a bunch of VMI calls - annotated with the call number (used to index into the ROM).

--Snip--

 55:   56                      push   %esi
 56:   e8 10 00 00 00          call   6b <.vmi.translation+0x6b>
 5b:   8d 64 24 04             lea    0x4(%esp,1),%esp

Here, a 4 argument hypercall, automatically emitted with regparm(3) convention and one stack argument.

And some use sites:

00000029 <default_idle>:

     29:       55                      push   %ebp
     2a:       89 e5                   mov    %esp,%ebp
     2c:       56                      push   %esi
     2d:       53                      push   %ebx
     2e:       fb                      sti   
     2f:       66                      data16
     30:       66                      data16
     31:       66                      data16
     32:       90                      nop

Note the automatic nop padding, out to the size of the emitted translation (5 byte call sequence). And here, in switch to, the automatically generated padding for the calls in load_TLS (VMI_WriteGDTEntry, the 4 argument call shown above). Note the perfect match between the 10 byte call sequence above and this code.

    b39:       89 0c d0                mov    %ecx,(%eax,%edx,8)
    b3c:       89 74 d0 04             mov    %esi,0x4(%eax,%edx,8)
    b40:       66                      data16
    b41:       66                      data16
    b42:       90                      nop   

The annotation table is less useful to look at in this form, but encodes the EIPs of the native sequences as well as the translations:

00000000 <.vmi.annotation>:

  0:   23 00                   and    (%eax),%eax
  2:   00 00                   add    %al,(%eax)
  4:   2e 00 00                add    %al,%cs:(%eax)
  7:   00 00                   add    %al,(%eax)
  9:   00 00                   add    %al,(%eax)
  b:   00 05 05 04 00 24       add    %al,0x24000405
 11:   00 00                   add    %al,(%eax)
 13:   00 62 00                add    %ah,0x0(%edx)
 16:   00 00                   add    %al,(%eax)
 18:   05 00 00 00 05          add    $0x5000000,%eax