Skip to contentSkip to conversation

x86 vdso32 cleanups

PATCH In review Brian Gerst 2 participants 14 messages Aug 29, 2015 – Sep 1, 2015 1 message not in the archive

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover x86 vdso32 cleanups

This patch set contains several cleanups to the 32-bit VDSO. The main change is to only build one VDSO image, and select the syscall entry point at runtime.

 arch/x86/entry/vdso/.gitignore                 |  4 +---
 arch/x86/entry/vdso/Makefile                   | 53 ++++++++++++++++++++++-------------------------------
 arch/x86/entry/vdso/{vdso32 => }/int80.S       | 13 +------------
 arch/x86/entry/vdso/{vdso32 => }/sigreturn.S   |  9 +++++++--
 arch/x86/entry/vdso/{vdso32 => }/syscall.S     | 23 +++++------------------
 arch/x86/entry/vdso/{vdso32 => }/sysenter.S    | 19 +++++--------------
 arch/x86/entry/vdso/vclock_gettime.c           | 31 +++++++++++++++++++++++++++++++
 arch/x86/entry/vdso/vdso-note.S                | 32 +++++++++++++++++++++++++++++++-
 arch/x86/entry/vdso/vdso2c.c                   |  2 ++
 arch/x86/entry/vdso/vdso32-setup.c             | 15 ++++++++-------
 arch/x86/entry/vdso/{vdso32 => }/vdso32.lds.S  |  2 +-
 arch/x86/entry/vdso/vdso32/.gitignore          |  1 -
 arch/x86/entry/vdso/vdso32/note.S              | 44 --------------------------------------------
 arch/x86/entry/vdso/vdso32/vclock_gettime.c    | 30 ------------------------------
 arch/x86/entry/vdso/vdso32/vdso-fakesections.c |  1 -
 arch/x86/entry/vdso/vma.c                      |  6 +++---
 arch/x86/ia32/ia32_signal.c                    |  4 ++--
 arch/x86/include/asm/elf.h                     |  3 +--
 arch/x86/include/asm/vdso.h                    | 20 +++++++++++++-------
 arch/x86/kernel/signal.c                       |  4 ++--
 arch/x86/xen/setup.c                           | 13 ++-----------
 arch/x86/xen/vdso.h                            |  4 ----
 22 files changed, 137 insertions(+), 196 deletions(-)
#1 lore raw

PATCH 1/7 x86/vdso32: Separate sigreturn code

Compile a separate sigreturn.o instead of including it in the three syscall entry stub files. Use alternatives to patch in a syscall instruction when supported.

---
 arch/x86/entry/vdso/Makefile           | 3 ++-
 arch/x86/entry/vdso/vdso32/int80.S     | 5 +----
 arch/x86/entry/vdso/vdso32/sigreturn.S | 9 +++++++--
 arch/x86/entry/vdso/vdso32/syscall.S   | 7 +------
 arch/x86/entry/vdso/vdso32/sysenter.S  | 5 +----
 5 files changed, 12 insertions(+), 17 deletions(-)
arch/x86/entry/vdso/Makefile +2 -1
index a3d0767..b4cd431 100644
@@ -140,7 +140,7 @@ override obj-dirs = $(dir $(obj)) $(obj)/vdso32/
 
 targets += vdso32/vdso32.lds
 targets += vdso32/note.o vdso32/vclock_gettime.o $(vdso32.so-y:%=vdso32/%.o)
-targets += vdso32/vclock_gettime.o
+targets += vdso32/vclock_gettime.o vdso32/sigreturn.o
 
 $(obj)/vdso32.o: $(vdso32-images:%=$(obj)/%)
 
@@ -163,6 +163,7 @@ $(vdso32-images:%=$(obj)/%.dbg): $(obj)/vdso32-%.so.dbg: FORCE \
 				 $(obj)/vdso32/vdso32.lds \
 				 $(obj)/vdso32/vclock_gettime.o \
 				 $(obj)/vdso32/note.o \
+				 $(obj)/vdso32/sigreturn.o \
 				 $(obj)/vdso32/%.o
 	$(call if_changed,vdso)
 
arch/x86/entry/vdso/vdso32/int80.S +1 -4
index b15b7c0..e40af1c 100644
@@ -1,10 +1,7 @@
 /*
  * Code for the vDSO.  This version uses the old int $0x80 method.
- *
- * First get the common code for the sigreturn entry points.
- * This must come first.
  */
-#include "sigreturn.S"
+#include <linux/linkage.h>
 
 	.text
 	.globl __kernel_vsyscall
arch/x86/entry/vdso/vdso32/sigreturn.S +7 -2
index d7ec4e2..ca0e6ca 100644
@@ -9,9 +9,14 @@
 #include <linux/linkage.h>
 #include <asm/unistd_32.h>
 #include <asm/asm-offsets.h>
+#include <asm/alternative-asm.h>
+#include <asm/cpufeature.h>
 
-#ifndef SYSCALL_ENTER_KERNEL
-#define	SYSCALL_ENTER_KERNEL	int $0x80
+#ifdef CONFIG_COMPAT
+#define SYSCALL_ENTER_KERNEL \
+	ALTERNATIVE "int $0x80", "syscall", X86_FEATURE_SYSCALL32
+#else
+#define SYSCALL_ENTER_KERNEL int $0x80
 #endif
 
 	.text
arch/x86/entry/vdso/vdso32/syscall.S +1 -6
index 6b286bb..75545ec 100644
@@ -1,12 +1,7 @@
 /*
  * Code for the vDSO.  This version uses the syscall instruction.
- *
- * First get the common code for the sigreturn entry points.
- * This must come first.
  */
-#define SYSCALL_ENTER_KERNEL	syscall
-#include "sigreturn.S"
-
+#include <linux/linkage.h>
 #include <asm/segment.h>
 
 	.text
arch/x86/entry/vdso/vdso32/sysenter.S +1 -4
index e354bce..e99c7699 100644
@@ -1,10 +1,7 @@
 /*
  * Code for the vDSO.  This version uses the sysenter instruction.
- *
- * First get the common code for the sigreturn entry points.
- * This must come first.
  */
-#include "sigreturn.S"
+#include <linux/linkage.h>
 
 /*
  * The caller puts arg2 in %ecx, which gets pushed. The kernel will use
Signature · 1 line
2.4.3
#2 lore raw

PATCH 2/7 x86/vdso32: Remove VDSO32_vsyscall_eh_frame_size

In reply to Brian Gerst

This symbol and the padding are unnecessary since we no longer rely on the symbols being exactly the same in each variant of the vdso32.

---
 arch/x86/entry/vdso/vdso32/int80.S    | 8 --------
 arch/x86/entry/vdso/vdso32/syscall.S  | 8 --------
 arch/x86/entry/vdso/vdso32/sysenter.S | 6 ------
 3 files changed, 22 deletions(-)
arch/x86/entry/vdso/vdso32/int80.S +0 -8
index e40af1c..667b25e 100644
@@ -43,11 +43,3 @@ __kernel_vsyscall:
 	.align 4
 .LENDFDEDLSI:
 	.previous
-
-	/*
-	 * Pad out the segment to match the size of the sysenter.S version.
-	 */
-VDSO32_vsyscall_eh_frame_size = 0x40
-	.section .data,"aw",@progbits
-	.space VDSO32_vsyscall_eh_frame_size-(.LENDFDEDLSI-.LSTARTFRAMEDLSI), 0
-	.previous
arch/x86/entry/vdso/vdso32/syscall.S +0 -8
index 75545ec..73f1428 100644
@@ -60,11 +60,3 @@ __kernel_vsyscall:
 	.align 4
 .LENDFDE1:
 	.previous
-
-	/*
-	 * Pad out the segment to match the size of the sysenter.S version.
-	 */
-VDSO32_vsyscall_eh_frame_size = 0x40
-	.section .data,"aw",@progbits
-	.space VDSO32_vsyscall_eh_frame_size-(.LENDFDE1-.LSTARTFRAME), 0
-	.previous
arch/x86/entry/vdso/vdso32/sysenter.S +0 -6
index e99c7699..e8e3080 100644
@@ -105,9 +105,3 @@ VDSO32_SYSENTER_RETURN:	/* Symbol used by sysenter.c via vdso32-syms.h */
 	.align 4
 .LENDFDEDLSI:
 	.previous
-
-	/*
-	 * Emit a symbol with the size of this .eh_frame data,
-	 * to verify it matches the other versions.
-	 */
-VDSO32_vsyscall_eh_frame_size = (.LENDFDEDLSI-.LSTARTFRAMEDLSI)
Signature · 1 line
2.4.3
#3 lore raw

PATCH 3/7 x86/vdso32: Remove unused vdso-fakesections.c

In reply to Brian Gerst
---
 arch/x86/entry/vdso/vdso32/vdso-fakesections.c | 1 -
 1 file changed, 1 deletion(-)
 delete mode 100644 arch/x86/entry/vdso/vdso32/vdso-fakesections.c
arch/x86/entry/vdso/vdso32/vdso-fakesections.c deleted +0 -1
deleted file mode 100644
index 541468e..0000000
@@ -1 +0,0 @@
-#include "../vdso-fakesections.c"
Signature · 1 line
2.4.3
#5 lore raw

PATCH 4/7 x86/vdso32: Build single vdso32 image

In reply to Brian Gerst

Currently, there are three images that are built for vdso32, differing only in the syscall entry code. The syscall entry is a tiny fraction of the total code, so most of the vdso code is duplicated in memory three times. This patch merges all the syscall entry points into one image, and instead of selecting the image, selects the entry point that is placed in the AT_SYSINFO vector and the ELF entry point.

---
 arch/x86/entry/vdso/.gitignore        |  3 ---
 arch/x86/entry/vdso/Makefile          | 44 ++++++++++++-----------------------
 arch/x86/entry/vdso/vdso2c.c          |  2 ++
 arch/x86/entry/vdso/vdso32-setup.c    | 15 ++++++------
 arch/x86/entry/vdso/vdso32/syscall.S  |  8 +++----
 arch/x86/entry/vdso/vdso32/sysenter.S |  8 +++----
 arch/x86/entry/vdso/vma.c             |  6 ++---
 arch/x86/ia32/ia32_signal.c           |  4 ++--
 arch/x86/include/asm/elf.h            |  3 +--
 arch/x86/include/asm/vdso.h           | 11 ++++-----
 arch/x86/kernel/signal.c              |  4 ++--
 arch/x86/xen/setup.c                  | 12 ++--------
 12 files changed, 47 insertions(+), 73 deletions(-)
arch/x86/entry/vdso/.gitignore +0 -3
index aae8ffd..a6a6ca8 100644
@@ -1,7 +1,4 @@
 vdso.lds
 vdsox32.lds
-vdso32-syscall-syms.lds
-vdso32-sysenter-syms.lds
-vdso32-int80-syms.lds
 vdso-image-*.c
 vdso2c
arch/x86/entry/vdso/Makefile +15 -29

72 lines not on this page · raw message

arch/x86/entry/vdso/vdso2c.c +2 -0
index 8627db2..cda5fa8 100644
@@ -100,6 +100,8 @@ struct vdso_sym required_syms[] = {
 	{"VDSO32_NOTE_MASK", true},
 	{"VDSO32_SYSENTER_RETURN", true},
 	{"__kernel_vsyscall", true},
+	{"__kernel_vsyscall_syscall", true},
+	{"__kernel_vsyscall_sysenter", true},
 	{"__kernel_sigreturn", true},
 	{"__kernel_rt_sigreturn", true},
 };
arch/x86/entry/vdso/vdso32-setup.c +8 -7
index e904c27..d644762 100644
@@ -10,6 +10,7 @@
 #include <linux/smp.h>
 #include <linux/kernel.h>
 #include <linux/mm_types.h>
+#include <linux/elf.h>
 
 #include <asm/cpufeature.h>
 #include <asm/processor.h>
@@ -60,23 +61,23 @@ __setup_param("vdso=", vdso_setup, vdso32_setup, 0);
 
 #endif	/* CONFIG_X86_64 */
 
-#if defined(CONFIG_X86_32) || defined(CONFIG_COMPAT)
-const struct vdso_image *selected_vdso32;
-#endif
+unsigned long selected_vsyscall;
 
 int __init sysenter_setup(void)
 {
 #ifdef CONFIG_COMPAT
 	if (vdso32_syscall())
-		selected_vdso32 = &vdso_image_32_syscall;
+		selected_vsyscall = vdso_image_32.sym___kernel_vsyscall_syscall;
 	else
 #endif
 	if (vdso32_sysenter())
-		selected_vdso32 = &vdso_image_32_sysenter;
+		selected_vsyscall = vdso_image_32.sym___kernel_vsyscall_sysenter;
 	else
-		selected_vdso32 = &vdso_image_32_int80;
+		selected_vsyscall = vdso_image_32.sym___kernel_vsyscall;
+
+	((struct elf32_hdr *)vdso_image_32.data)->e_entry = selected_vsyscall;
 
-	init_vdso_image(selected_vdso32);
+	init_vdso_image(&vdso_image_32);
 
 	return 0;
 }
arch/x86/entry/vdso/vdso32/syscall.S +4 -4
index 73f1428..50490c8 100644
@@ -5,10 +5,10 @@
 #include <asm/segment.h>
 
 	.text
-	.globl __kernel_vsyscall
-	.type __kernel_vsyscall,@function
+	.globl __kernel_vsyscall_syscall
+	.type __kernel_vsyscall_syscall,@function
 	ALIGN
-__kernel_vsyscall:
+__kernel_vsyscall_syscall:
 .LSTART_vsyscall:
 	push	%ebp
 .Lpush_ebp:
@@ -19,7 +19,7 @@ __kernel_vsyscall:
 .Lpop_ebp:
 	ret
 .LEND_vsyscall:
-	.size __kernel_vsyscall,.-.LSTART_vsyscall
+	.size __kernel_vsyscall_syscall,.-.LSTART_vsyscall
 
 	.section .eh_frame,"a",@progbits
 .LSTARTFRAME:
arch/x86/entry/vdso/vdso32/sysenter.S +4 -4
index e8e3080..458954a 100644
@@ -22,10 +22,10 @@
  * three words on the parent stack do not get copied to the child.
  */
 	.text
-	.globl __kernel_vsyscall
-	.type __kernel_vsyscall,@function
+	.globl __kernel_vsyscall_sysenter
+	.type __kernel_vsyscall_sysenter,@function
 	ALIGN
-__kernel_vsyscall:
+__kernel_vsyscall_sysenter:
 .LSTART_vsyscall:
 	push %ecx
 .Lpush_ecx:
@@ -51,7 +51,7 @@ VDSO32_SYSENTER_RETURN:	/* Symbol used by sysenter.c via vdso32-syms.h */
 .Lpop_ecx:
 	ret
 .LEND_vsyscall:
-	.size __kernel_vsyscall,.-.LSTART_vsyscall
+	.size __kernel_vsyscall_sysenter,.-.LSTART_vsyscall
 	.previous
 
 	.section .eh_frame,"a",@progbits
arch/x86/entry/vdso/vma.c +3 -3
index 4345431..c726d49 100644
@@ -185,14 +185,14 @@ static int load_vdso32(void)
 	if (vdso32_enabled != 1)  /* Other values all mean "disabled" */
 		return 0;
 
-	ret = map_vdso(selected_vdso32, false);
+	ret = map_vdso(&vdso_image_32, false);
 	if (ret)
 		return ret;
 
-	if (selected_vdso32->sym_VDSO32_SYSENTER_RETURN)
+	if (vdso_image_32.sym_VDSO32_SYSENTER_RETURN)
 		current_thread_info()->sysenter_return =
 			current->mm->context.vdso +
-			selected_vdso32->sym_VDSO32_SYSENTER_RETURN;
+			vdso_image_32.sym_VDSO32_SYSENTER_RETURN;
 
 	return 0;
 }
arch/x86/ia32/ia32_signal.c +2 -2
index a0a19b7..e6a5c275 100644
@@ -289,7 +289,7 @@ int ia32_setup_frame(int sig, struct ksignal *ksig,
 		/* Return stub is in 32bit vsyscall page */
 		if (current->mm->context.vdso)
 			restorer = current->mm->context.vdso +
-				selected_vdso32->sym___kernel_sigreturn;
+				vdso_image_32.sym___kernel_sigreturn;
 		else
 			restorer = &frame->retcode;
 	}
@@ -368,7 +368,7 @@ int ia32_setup_rt_frame(int sig, struct ksignal *ksig,
 			restorer = ksig->ka.sa.sa_restorer;
 		else
 			restorer = current->mm->context.vdso +
-				selected_vdso32->sym___kernel_rt_sigreturn;
+				vdso_image_32.sym___kernel_rt_sigreturn;
 		put_user_ex(ptr_to_compat(restorer), &frame->pretcode);
 
 		/*
arch/x86/include/asm/elf.h +1 -2
index 141c561..ccc1d31 100644
@@ -327,8 +327,7 @@ else									\
 #define VDSO_CURRENT_BASE	((unsigned long)current->mm->context.vdso)
 
 #define VDSO_ENTRY							\
-	((unsigned long)current->mm->context.vdso +			\
-	 selected_vdso32->sym___kernel_vsyscall)
+	((unsigned long)current->mm->context.vdso + selected_vsyscall)
 
 struct linux_binprm;
 
arch/x86/include/asm/vdso.h +4 -7
index 8021bd2..16d5c18 100644
@@ -26,6 +26,8 @@ struct vdso_image {
 	long sym___kernel_sigreturn;
 	long sym___kernel_rt_sigreturn;
 	long sym___kernel_vsyscall;
+	long sym___kernel_vsyscall_syscall;
+	long sym___kernel_vsyscall_sysenter;
 	long sym_VDSO32_SYSENTER_RETURN;
 };
 
@@ -38,13 +40,8 @@ extern const struct vdso_image vdso_image_x32;
 #endif
 
 #if defined CONFIG_X86_32 || defined CONFIG_COMPAT
-extern const struct vdso_image vdso_image_32_int80;
-#ifdef CONFIG_COMPAT
-extern const struct vdso_image vdso_image_32_syscall;
-#endif
-extern const struct vdso_image vdso_image_32_sysenter;
-
-extern const struct vdso_image *selected_vdso32;
+extern const struct vdso_image vdso_image_32;
+extern unsigned long selected_vsyscall;
 #endif
 
 extern void __init init_vdso_image(const struct vdso_image *image);
arch/x86/kernel/signal.c +2 -2
index da52e6b..d87ce92 100644
@@ -299,7 +299,7 @@ __setup_frame(int sig, struct ksignal *ksig, sigset_t *set,
 
 	if (current->mm->context.vdso)
 		restorer = current->mm->context.vdso +
-			selected_vdso32->sym___kernel_sigreturn;
+			vdso_image_32.sym___kernel_sigreturn;
 	else
 		restorer = &frame->retcode;
 	if (ksig->ka.sa.sa_flags & SA_RESTORER)
@@ -363,7 +363,7 @@ static int __setup_rt_frame(int sig, struct ksignal *ksig,
 
 		/* Set up to return from userspace.  */
 		restorer = current->mm->context.vdso +
-			selected_vdso32->sym___kernel_rt_sigreturn;
+			vdso_image_32.sym___kernel_rt_sigreturn;
 		if (ksig->ka.sa.sa_flags & SA_RESTORER)
 			restorer = ksig->ka.sa.sa_restorer;
 		put_user_ex(restorer, &frame->pretcode);
arch/x86/xen/setup.c +2 -10
index 55f388e..b166ffd 100644
@@ -753,17 +753,9 @@ char * __init xen_auto_xlated_memory_setup(void)
 static void __init fiddle_vdso(void)
 {
 #ifdef CONFIG_X86_32
-	/*
-	 * This could be called before selected_vdso32 is initialized, so
-	 * just fiddle with both possible images.  vdso_image_32_syscall
-	 * can't be selected, since it only exists on 64-bit systems.
-	 */
 	u32 *mask;
-	mask = vdso_image_32_int80.data +
-		vdso_image_32_int80.sym_VDSO32_NOTE_MASK;
-	*mask |= 1 << VDSO_NOTE_NONEGSEG_BIT;
-	mask = vdso_image_32_sysenter.data +
-		vdso_image_32_sysenter.sym_VDSO32_NOTE_MASK;
+	mask = vdso_image_32.data +
+		vdso_image_32.sym_VDSO32_NOTE_MASK;
 	*mask |= 1 << VDSO_NOTE_NONEGSEG_BIT;
 #endif
 }
Signature · 1 line
2.4.3
#6 lore raw

PATCH 5/7 x86/vdso: Merge 32-bit and 64-bit source files

In reply to Brian Gerst

Merge the 32-bit versions of vclock_gettime and note.S into the 64-bit code. Add some make rules to handle the combined code.

---
 arch/x86/entry/vdso/Makefile                | 10 ++++++-
 arch/x86/entry/vdso/vclock_gettime.c        | 31 ++++++++++++++++++++
 arch/x86/entry/vdso/vdso-note.S             | 34 +++++++++++++++++++++-
 arch/x86/entry/vdso/vdso32/note.S           | 44 -----------------------------
 arch/x86/entry/vdso/vdso32/vclock_gettime.c | 30 --------------------
 5 files changed, 73 insertions(+), 76 deletions(-)
 delete mode 100644 arch/x86/entry/vdso/vdso32/note.S
 delete mode 100644 arch/x86/entry/vdso/vdso32/vclock_gettime.c
arch/x86/entry/vdso/Makefile +9 -1
index 282121a..a8aa0c0 100644
@@ -128,6 +128,7 @@ VDSO_LDFLAGS_vdso32.lds = -m32 -Wl,-m,elf_i386 -Wl,-soname=linux-gate.so.1
 override obj-dirs = $(dir $(obj)) $(obj)/vdso32/
 
 KBUILD_AFLAGS_32 := $(filter-out -m64,$(KBUILD_AFLAGS))
+KBUILD_AFLAGS_32 += -DBUILD_VDSO32
 $(obj)/vdso32.so.dbg: KBUILD_AFLAGS = $(KBUILD_AFLAGS_32)
 $(obj)/vdso32.so.dbg: asflags-$(CONFIG_X86_64) += -m32
 
@@ -140,8 +141,9 @@ KBUILD_CFLAGS_32 += $(call cc-option, -fno-stack-protector)
 KBUILD_CFLAGS_32 += $(call cc-option, -foptimize-sibling-calls)
 KBUILD_CFLAGS_32 += -fno-omit-frame-pointer
 KBUILD_CFLAGS_32 += -DDISABLE_BRANCH_PROFILING
+KBUILD_CFLAGS_32 += -DBUILD_VDSO32
 
-vobjs32-y := vdso32/vclock_gettime.o vdso32/note.o vdso32/sigreturn.o
+vobjs32-y := vclock_gettime-32.o vdso-note-32.o vdso32/sigreturn.o
 vobjs32-y += vdso32/int80.o vdso32/sysenter.o
 vobjs32-$(CONFIG_COMPAT) += vdso32/syscall.o
 
@@ -149,6 +151,12 @@ vobjs32 := $(foreach F,$(vobjs32-y),$(obj)/$F)
 
 targets += vdso32/vdso32.lds $(vobjs32-y)
 
+$(obj)/%-32.o: $(src)/%.c FORCE
+	$(call if_changed_dep,cc_o_c)
+
+$(obj)/%-32.o: $(src)/%.S FORCE
+	$(call if_changed_dep,as_o_S)
+
 $(obj)/vdso32.so.dbg: KBUILD_CFLAGS = $(KBUILD_CFLAGS_32)
 $(obj)/vdso32.so.dbg: $(obj)/vdso32/vdso32.lds $(vobjs32) FORCE
 	$(call if_changed,vdso)
arch/x86/entry/vdso/vclock_gettime.c +31 -0
index ca94fa6..0d1faee 100644
@@ -11,6 +11,37 @@
  * Check with readelf after changing.
  */
 
+#ifdef BUILD_VDSO32
+
+#ifndef CONFIG_CC_OPTIMIZE_FOR_SIZE
+#undef CONFIG_OPTIMIZE_INLINING
+#endif
+
+#undef CONFIG_X86_PPRO_FENCE
+
+#ifdef CONFIG_X86_64
+
+/*
+ * in case of a 32 bit VDSO for a 64 bit kernel fake a 32 bit kernel
+ * configuration
+ */
+#undef CONFIG_64BIT
+#undef CONFIG_X86_64
+#undef CONFIG_ILLEGAL_POINTER_VALUE
+#undef CONFIG_SPARSEMEM_VMEMMAP
+#undef CONFIG_NR_CPUS
+
+#define CONFIG_X86_32 1
+#define CONFIG_PAGE_OFFSET 0
+#define CONFIG_ILLEGAL_POINTER_VALUE 0
+#define CONFIG_NR_CPUS 1
+
+#define BUILD_VDSO32_64
+
+#endif /* CONFIG_X86_64 */
+
+#endif /* BUILD_VDSO32 */
+
 #include <uapi/linux/time.h>
 #include <asm/vgtod.h>
 #include <asm/hpet.h>
arch/x86/entry/vdso/vdso-note.S +33 -1
index 79a071e..eb8a6c7 100644
@@ -3,10 +3,42 @@
  * Here we can supply some information useful to userland.
  */
 
-#include <linux/uts.h>
 #include <linux/version.h>
 #include <linux/elfnote.h>
 
+/* Ideally this would use UTS_NAME, but using a quoted string here
+   doesn't work. Remember to change this when changing the
+   kernel's name. */
 ELFNOTE_START(Linux, 0, "a")
 	.long LINUX_VERSION_CODE
 ELFNOTE_END
+
+#if defined(CONFIG_XEN) && defined(BUILD_VDSO32)
+/*
+ * Add a special note telling glibc's dynamic linker a fake hardware
+ * flavor that it will use to choose the search path for libraries in the
+ * same way it uses real hardware capabilities like "mmx".
+ * We supply "nosegneg" as the fake capability, to indicate that we
+ * do not like negative offsets in instructions using segment overrides,
+ * since we implement those inefficiently.  This makes it possible to
+ * install libraries optimized to avoid those access patterns in someplace
+ * like /lib/i686/tls/nosegneg.  Note that an /etc/ld.so.conf.d/file
+ * corresponding to the bits here is needed to make ldconfig work right.
+ * It should contain:
+ *	hwcap 1 nosegneg
+ * to match the mapping of bit to name that we give here.
+ *
+ * At runtime, the fake hardware feature will be considered to be present
+ * if its bit is set in the mask word.  So, we start with the mask 0, and
+ * at boot time we set VDSO_NOTE_NONEGSEG_BIT if running under Xen.
+ */
+
+#include "../../xen/vdso.h"	/* Defines VDSO_NOTE_NONEGSEG_BIT.  */
+
+ELFNOTE_START(GNU, 2, "a")
+	.long 1			/* ncaps */
+VDSO32_NOTE_MASK:		/* Symbol used by arch/x86/xen/setup.c */
+	.long 0			/* mask */
+	.byte VDSO_NOTE_NONEGSEG_BIT; .asciz "nosegneg"	/* bit, name */
+ELFNOTE_END
+#endif
arch/x86/entry/vdso/vdso32/note.S deleted +0 -44

44 lines not on this page · raw message

arch/x86/entry/vdso/vdso32/vclock_gettime.c deleted +0 -30
deleted file mode 100644
index 175cc72..0000000
@@ -1,30 +0,0 @@
-#define BUILD_VDSO32
-
-#ifndef CONFIG_CC_OPTIMIZE_FOR_SIZE
-#undef CONFIG_OPTIMIZE_INLINING
-#endif
-
-#undef CONFIG_X86_PPRO_FENCE
-
-#ifdef CONFIG_X86_64
-
-/*
- * in case of a 32 bit VDSO for a 64 bit kernel fake a 32 bit kernel
- * configuration
- */
-#undef CONFIG_64BIT
-#undef CONFIG_X86_64
-#undef CONFIG_ILLEGAL_POINTER_VALUE
-#undef CONFIG_SPARSEMEM_VMEMMAP
-#undef CONFIG_NR_CPUS
-
-#define CONFIG_X86_32 1
-#define CONFIG_PAGE_OFFSET 0
-#define CONFIG_ILLEGAL_POINTER_VALUE 0
-#define CONFIG_NR_CPUS 1
-
-#define BUILD_VDSO32_64
-
-#endif
-
-#include "../vclock_gettime.c"
Signature · 1 line
2.4.3
#7 lore raw

PATCH 6/7 x86/vdso32/xen: Move VDSO_NOTE_NONEGSEG_BIT define

In reply to Brian Gerst

Xen had its own vdso.h just to define VDSO_NOTE_NONEGSEG_BIT. Move it to the main vdso.h.

---
 arch/x86/entry/vdso/vdso-note.S | 4 +---
 arch/x86/include/asm/vdso.h     | 9 +++++++++
 arch/x86/xen/setup.c            | 1 -
 arch/x86/xen/vdso.h             | 4 ----
 4 files changed, 10 insertions(+), 8 deletions(-)
 delete mode 100644 arch/x86/xen/vdso.h
arch/x86/entry/vdso/vdso-note.S +1 -3
index eb8a6c7..34aa574 100644
@@ -5,6 +5,7 @@
 
 #include <linux/version.h>
 #include <linux/elfnote.h>
+#include <asm/vdso.h>
 
 /* Ideally this would use UTS_NAME, but using a quoted string here
    doesn't work. Remember to change this when changing the
@@ -32,9 +33,6 @@ ELFNOTE_END
  * if its bit is set in the mask word.  So, we start with the mask 0, and
  * at boot time we set VDSO_NOTE_NONEGSEG_BIT if running under Xen.
  */
-
-#include "../../xen/vdso.h"	/* Defines VDSO_NOTE_NONEGSEG_BIT.  */
-
 ELFNOTE_START(GNU, 2, "a")
 	.long 1			/* ncaps */
 VDSO32_NOTE_MASK:		/* Symbol used by arch/x86/xen/setup.c */
arch/x86/include/asm/vdso.h +9 -0
index 16d5c18..8d9a961 100644
@@ -48,4 +48,13 @@ extern void __init init_vdso_image(const struct vdso_image *image);
 
 #endif /* __ASSEMBLER__ */
 
+#if defined(CONFIG_X86_32) || defined(CONFIG_IA32_EMULATION)
+/*
+ * Bit used for the pseudo-hwcap for non-negative segments.  We use
+ *    bit 1 to avoid bugs in some versions of glibc when bit 0 is
+ *       used; the choice is otherwise arbitrary.
+ */
+#define VDSO_NOTE_NONEGSEG_BIT  1
+#endif
+
 #endif /* _ASM_X86_VDSO_H */
arch/x86/xen/setup.c +0 -1
index b166ffd..79f9ed7 100644
@@ -28,7 +28,6 @@
 #include <xen/interface/physdev.h>
 #include <xen/features.h>
 #include "xen-ops.h"
-#include "vdso.h"
 #include "p2m.h"
 #include "mmu.h"
 
arch/x86/xen/vdso.h deleted +0 -4
deleted file mode 100644
index 861fedf..0000000
@@ -1,4 +0,0 @@
-/* Bit used for the pseudo-hwcap for non-negative segments.  We use
-   bit 1 to avoid bugs in some versions of glibc when bit 0 is
-   used; the choice is otherwise arbitrary. */
-#define VDSO_NOTE_NONEGSEG_BIT	1
Signature · 1 line
2.4.3
#8 lore raw

PATCH 7/7 x86/vdso32: Remove vdso32 subdirectory

In reply to Brian Gerst

Since the vdso32 subdirectory doesn't have a proper Makefile, it is more difficult to work with. Move the remaining files up one level.

---
 arch/x86/entry/vdso/.gitignore          |   1 +
 arch/x86/entry/vdso/Makefile            |  14 ++-
 arch/x86/entry/vdso/int80.S             |  45 ++++++++++
 arch/x86/entry/vdso/sigreturn.S         | 150 ++++++++++++++++++++++++++++++++
 arch/x86/entry/vdso/syscall.S           |  62 +++++++++++++
 arch/x86/entry/vdso/sysenter.S          | 107 +++++++++++++++++++++++
 arch/x86/entry/vdso/vdso32.lds.S        |  37 ++++++++
 arch/x86/entry/vdso/vdso32/.gitignore   |   1 -
 arch/x86/entry/vdso/vdso32/int80.S      |  45 ----------
 arch/x86/entry/vdso/vdso32/sigreturn.S  | 150 --------------------------------
 arch/x86/entry/vdso/vdso32/syscall.S    |  62 -------------
 arch/x86/entry/vdso/vdso32/sysenter.S   | 107 -----------------------
 arch/x86/entry/vdso/vdso32/vdso32.lds.S |  37 --------
 13 files changed, 407 insertions(+), 411 deletions(-)
 create mode 100644 arch/x86/entry/vdso/int80.S
 create mode 100644 arch/x86/entry/vdso/sigreturn.S
 create mode 100644 arch/x86/entry/vdso/syscall.S
 create mode 100644 arch/x86/entry/vdso/sysenter.S
 create mode 100644 arch/x86/entry/vdso/vdso32.lds.S
 delete mode 100644 arch/x86/entry/vdso/vdso32/.gitignore
 delete mode 100644 arch/x86/entry/vdso/vdso32/int80.S
 delete mode 100644 arch/x86/entry/vdso/vdso32/sigreturn.S
 delete mode 100644 arch/x86/entry/vdso/vdso32/syscall.S
 delete mode 100644 arch/x86/entry/vdso/vdso32/sysenter.S
 delete mode 100644 arch/x86/entry/vdso/vdso32/vdso32.lds.S
arch/x86/entry/vdso/.gitignore +1 -0
index a6a6ca8..d285fe6 100644
@@ -1,4 +1,5 @@
 vdso.lds
 vdsox32.lds
+vdso32.lds
 vdso-image-*.c
 vdso2c
arch/x86/entry/vdso/Makefile +5 -9
index a8aa0c0..45df51e 100644
@@ -123,10 +123,6 @@ $(obj)/vdsox32.so.dbg: $(src)/vdsox32.lds $(vobjx32s) FORCE
 CPPFLAGS_vdso32.lds = $(CPPFLAGS_vdso.lds)
 VDSO_LDFLAGS_vdso32.lds = -m32 -Wl,-m,elf_i386 -Wl,-soname=linux-gate.so.1
 
-# This makes sure the $(obj) subdirectory exists even though vdso32/
-# is not a kbuild sub-make subdirectory.
-override obj-dirs = $(dir $(obj)) $(obj)/vdso32/
-
 KBUILD_AFLAGS_32 := $(filter-out -m64,$(KBUILD_AFLAGS))
 KBUILD_AFLAGS_32 += -DBUILD_VDSO32
 $(obj)/vdso32.so.dbg: KBUILD_AFLAGS = $(KBUILD_AFLAGS_32)
@@ -143,13 +139,13 @@ KBUILD_CFLAGS_32 += -fno-omit-frame-pointer
 KBUILD_CFLAGS_32 += -DDISABLE_BRANCH_PROFILING
 KBUILD_CFLAGS_32 += -DBUILD_VDSO32
 
-vobjs32-y := vclock_gettime-32.o vdso-note-32.o vdso32/sigreturn.o
-vobjs32-y += vdso32/int80.o vdso32/sysenter.o
-vobjs32-$(CONFIG_COMPAT) += vdso32/syscall.o
+vobjs32-y := vclock_gettime-32.o vdso-note-32.o sigreturn.o
+vobjs32-y += int80.o sysenter.o
+vobjs32-$(CONFIG_COMPAT) += syscall.o
 
 vobjs32 := $(foreach F,$(vobjs32-y),$(obj)/$F)
 
-targets += vdso32/vdso32.lds $(vobjs32-y)
+targets += vdso32.lds $(vobjs32-y)
 
 $(obj)/%-32.o: $(src)/%.c FORCE
 	$(call if_changed_dep,cc_o_c)
@@ -158,7 +154,7 @@ $(obj)/%-32.o: $(src)/%.S FORCE
 	$(call if_changed_dep,as_o_S)
 
 $(obj)/vdso32.so.dbg: KBUILD_CFLAGS = $(KBUILD_CFLAGS_32)
-$(obj)/vdso32.so.dbg: $(obj)/vdso32/vdso32.lds $(vobjs32) FORCE
+$(obj)/vdso32.so.dbg: $(obj)/vdso32.lds $(vobjs32) FORCE
 	$(call if_changed,vdso)
 
 #
arch/x86/entry/vdso/int80.S new file +45 -0

45 lines not on this page · raw message

arch/x86/entry/vdso/sigreturn.S new file +150 -0

150 lines not on this page · raw message

arch/x86/entry/vdso/syscall.S new file +62 -0

62 lines not on this page · raw message

arch/x86/entry/vdso/sysenter.S new file +107 -0

107 lines not on this page · raw message

arch/x86/entry/vdso/vdso32.lds.S new file +37 -0
new file mode 100644
index 0000000..d90e607
@@ -0,0 +1,37 @@
+/*
+ * Linker script for 32-bit vDSO.
+ * We #include the file to define the layout details.
+ *
+ * This file defines the version script giving the user-exported symbols in
+ * the DSO.
+ */
+
+#include <asm/page.h>
+
+#define BUILD_VDSO32
+
+#include "vdso-layout.lds.S"
+
+/* The ELF entry point can be used to set the AT_SYSINFO value.  */
+ENTRY(__kernel_vsyscall);
+
+/*
+ * This controls what userland symbols we export from the vDSO.
+ */
+VERSION
+{
+	LINUX_2.6 {
+	global:
+		__vdso_clock_gettime;
+		__vdso_gettimeofday;
+		__vdso_time;
+	};
+
+	LINUX_2.5 {
+	global:
+		__kernel_vsyscall;
+		__kernel_sigreturn;
+		__kernel_rt_sigreturn;
+	local: *;
+	};
+}
arch/x86/entry/vdso/vdso32/.gitignore deleted +0 -1
deleted file mode 100644
index e45fba9..0000000
@@ -1 +0,0 @@
-vdso32.lds
arch/x86/entry/vdso/vdso32/int80.S deleted +0 -45

45 lines not on this page · raw message

arch/x86/entry/vdso/vdso32/sigreturn.S deleted +0 -150

150 lines not on this page · raw message

arch/x86/entry/vdso/vdso32/syscall.S deleted +0 -62

62 lines not on this page · raw message

arch/x86/entry/vdso/vdso32/sysenter.S deleted +0 -107

107 lines not on this page · raw message

arch/x86/entry/vdso/vdso32/vdso32.lds.S deleted +0 -37
deleted file mode 100644
index 31056cf..0000000
@@ -1,37 +0,0 @@
-/*
- * Linker script for 32-bit vDSO.
- * We #include the file to define the layout details.
- *
- * This file defines the version script giving the user-exported symbols in
- * the DSO.
- */
-
-#include <asm/page.h>
-
-#define BUILD_VDSO32
-
-#include "../vdso-layout.lds.S"
-
-/* The ELF entry point can be used to set the AT_SYSINFO value.  */
-ENTRY(__kernel_vsyscall);
-
-/*
- * This controls what userland symbols we export from the vDSO.
- */
-VERSION
-{
-	LINUX_2.6 {
-	global:
-		__vdso_clock_gettime;
-		__vdso_gettimeofday;
-		__vdso_time;
-	};
-
-	LINUX_2.5 {
-	global:
-		__kernel_vsyscall;
-		__kernel_sigreturn;
-		__kernel_rt_sigreturn;
-	local: *;
-	};
-}
Signature · 1 line
2.4.3
#9 lore raw
In reply to Brian Gerst
Quoted 3 lines · Brian Gerst

This patch set contains several cleanups to the 32-bit VDSO. The main change is to only build one VDSO image, and select the syscall entry point at runtime.

Oh no, we have dueling patches!

I have a 2/3 finished series that cleans up the AT_SYSINFO mess differently, as I outlined earlier. I've only done the compat and common bits (no 32-bit native support quite yet), and it enters successfully on Intel using SYSENTER and on (fake) AMD using SYSCALL. The SYSRET bit isn't there yet.

Other than some ifdeffery, the final system_call.S looks like this:

https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/tree/arch/x86/entry/vdso/vdso32/system_call.S?h=x86/entry_compat

The meat is (sorry for whitespace damage):

.text
.globl __kernel_vsyscall
.type __kernel_vsyscall,@function
ALIGN
__kernel_vsyscall:
CFI_STARTPROC
/*
* Reshuffle regs so that all of any of the entry instructions
* will preserve enough state.
*/
pushl %edx
CFI_ADJUST_CFA_OFFSET 4
CFI_REL_OFFSET edx, 0
pushl %ecx
CFI_ADJUST_CFA_OFFSET 4
CFI_REL_OFFSET ecx, 0
movl %esp, %ecx

#ifdef CONFIG_X86_64
/* If SYSENTER is available, use it. */

ALTERNATIVE_2 "", "sysenter", X86_FEATURE_SYSENTER32, \
                 "syscall",  X86_FEATURE_SYSCALL32

#endif

/* Enter using int $0x80 */
movl (%esp), %ecx
int $0x80
GLOBAL(int80_landing_pad)

/* Restore ECX and EDX in case they were clobbered. */ popl %ecx
CFI_RESTORE ecx
CFI_ADJUST_CFA_OFFSET -4
popl %edx
CFI_RESTORE edx
CFI_ADJUST_CFA_OFFSET -4
ret
CFI_ENDPROC

.size __kernel_vsyscall,.-__kernel_vsyscall
.previous

And that's it.

What do you think? This comes with massively cleaned up kernel-side asm as well as a test case that actually validates the CFI directives.

Certainly, a bunch of your patches make sense regardless, and I'll review them and add them to my queue soon.

--Andy

#10 lore raw
Quoted 63 lines · Andy Lutomirski
Quoted 3 lines · Brian Gerst

This patch set contains several cleanups to the 32-bit VDSO. The main change is to only build one VDSO image, and select the syscall entry point at runtime.

Oh no, we have dueling patches!

I have a 2/3 finished series that cleans up the AT_SYSINFO mess differently, as I outlined earlier. I've only done the compat and common bits (no 32-bit native support quite yet), and it enters successfully on Intel using SYSENTER and on (fake) AMD using SYSCALL. The SYSRET bit isn't there yet.

Other than some ifdeffery, the final system_call.S looks like this:

https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/tree/arch/x86/entry/vdso/vdso32/system_call.S?h=x86/entry_compat

The meat is (sorry for whitespace damage):

.text
.globl __kernel_vsyscall
.type __kernel_vsyscall,@function
ALIGN
__kernel_vsyscall:
CFI_STARTPROC
/*
* Reshuffle regs so that all of any of the entry instructions
* will preserve enough state.
*/
pushl %edx
CFI_ADJUST_CFA_OFFSET 4
CFI_REL_OFFSET edx, 0
pushl %ecx
CFI_ADJUST_CFA_OFFSET 4
CFI_REL_OFFSET ecx, 0
movl %esp, %ecx

#ifdef CONFIG_X86_64
/* If SYSENTER is available, use it. */

ALTERNATIVE_2 "", "sysenter", X86_FEATURE_SYSENTER32, \
                 "syscall",  X86_FEATURE_SYSCALL32

#endif

/* Enter using int $0x80 */
movl (%esp), %ecx
int $0x80
GLOBAL(int80_landing_pad)

/* Restore ECX and EDX in case they were clobbered. */ popl %ecx
CFI_RESTORE ecx
CFI_ADJUST_CFA_OFFSET -4
popl %edx
CFI_RESTORE edx
CFI_ADJUST_CFA_OFFSET -4
ret
CFI_ENDPROC

.size __kernel_vsyscall,.-__kernel_vsyscall
.previous

And that's it.

Quoted 7 lines

What do you think? This comes with massively cleaned up kernel-side asm as well as a test case that actually validates the CFI directives.

Certainly, a bunch of your patches make sense regardless, and I'll review them and add them to my queue soon.

--Andy

How does the performance compare to the original? Looking at the disassembly, there are two added function calls, and it reloads the args from the stack instead of just shuffling registers.

--
Brian Gerst

#11 lore raw
Quoted 67 lines · Brian Gerst
Quoted 66 lines · Andy Lutomirski
Quoted 3 lines · Brian Gerst

This patch set contains several cleanups to the 32-bit VDSO. The main change is to only build one VDSO image, and select the syscall entry point at runtime.

Oh no, we have dueling patches!

I have a 2/3 finished series that cleans up the AT_SYSINFO mess differently, as I outlined earlier. I've only done the compat and common bits (no 32-bit native support quite yet), and it enters successfully on Intel using SYSENTER and on (fake) AMD using SYSCALL. The SYSRET bit isn't there yet.

Other than some ifdeffery, the final system_call.S looks like this:

https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/tree/arch/x86/entry/vdso/vdso32/system_call.S?h=x86/entry_compat

The meat is (sorry for whitespace damage):

.text
.globl __kernel_vsyscall
.type __kernel_vsyscall,@function
ALIGN
__kernel_vsyscall:
CFI_STARTPROC
/*
* Reshuffle regs so that all of any of the entry instructions
* will preserve enough state.
*/
pushl %edx
CFI_ADJUST_CFA_OFFSET 4
CFI_REL_OFFSET edx, 0
pushl %ecx
CFI_ADJUST_CFA_OFFSET 4
CFI_REL_OFFSET ecx, 0
movl %esp, %ecx

#ifdef CONFIG_X86_64
/* If SYSENTER is available, use it. */

ALTERNATIVE_2 "", "sysenter", X86_FEATURE_SYSENTER32, \
                 "syscall",  X86_FEATURE_SYSCALL32

#endif

/* Enter using int $0x80 */
movl (%esp), %ecx
int $0x80
GLOBAL(int80_landing_pad)

/* Restore ECX and EDX in case they were clobbered. */ popl %ecx
CFI_RESTORE ecx
CFI_ADJUST_CFA_OFFSET -4
popl %edx
CFI_RESTORE edx
CFI_ADJUST_CFA_OFFSET -4
ret
CFI_ENDPROC

.size __kernel_vsyscall,.-__kernel_vsyscall
.previous

And that's it.

What do you think? This comes with massively cleaned up kernel-side asm as well as a test case that actually validates the CFI directives.

Quoted 8 lines
Quoted 4 lines

Certainly, a bunch of your patches make sense regardless, and I'll review them and add them to my queue soon.

--Andy

How does the performance compare to the original? Looking at the disassembly, there are two added function calls, and it reloads the args from the stack instead of just shuffling registers.

The replacement is dramatically faster, which means I probably benchmarked it wrong. I'll try again in a day or two.

--Andy

A message that is not in the archive (sent off-list or not yet received): CALCETrVdaDuSuLLByWbFsEei6LYaxZK6qFtJAHZhRu3ZqUc+RQ@mail.gmail.com

#13 lore raw
Quoted 80 lines · Andy Lutomirski
Quoted 78 lines · Andy Lutomirski
Quoted 75 lines · Brian Gerst
Quoted 63 lines · Andy Lutomirski
Quoted 3 lines · Brian Gerst

This patch set contains several cleanups to the 32-bit VDSO. The main change is to only build one VDSO image, and select the syscall entry point at runtime.

Oh no, we have dueling patches!

I have a 2/3 finished series that cleans up the AT_SYSINFO mess differently, as I outlined earlier. I've only done the compat and common bits (no 32-bit native support quite yet), and it enters successfully on Intel using SYSENTER and on (fake) AMD using SYSCALL. The SYSRET bit isn't there yet.

Other than some ifdeffery, the final system_call.S looks like this:

https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/tree/arch/x86/entry/vdso/vdso32/system_call.S?h=x86/entry_compat

The meat is (sorry for whitespace damage):

.text
.globl __kernel_vsyscall
.type __kernel_vsyscall,@function
ALIGN
__kernel_vsyscall:
CFI_STARTPROC
/*
* Reshuffle regs so that all of any of the entry instructions
* will preserve enough state.
*/
pushl %edx
CFI_ADJUST_CFA_OFFSET 4
CFI_REL_OFFSET edx, 0
pushl %ecx
CFI_ADJUST_CFA_OFFSET 4
CFI_REL_OFFSET ecx, 0
movl %esp, %ecx

#ifdef CONFIG_X86_64
/* If SYSENTER is available, use it. */

ALTERNATIVE_2 "", "sysenter", X86_FEATURE_SYSENTER32, \
                 "syscall",  X86_FEATURE_SYSCALL32

#endif

/* Enter using int $0x80 */
movl (%esp), %ecx
int $0x80
GLOBAL(int80_landing_pad)

/* Restore ECX and EDX in case they were clobbered. */ popl %ecx
CFI_RESTORE ecx
CFI_ADJUST_CFA_OFFSET -4
popl %edx
CFI_RESTORE edx
CFI_ADJUST_CFA_OFFSET -4
ret
CFI_ENDPROC

.size __kernel_vsyscall,.-__kernel_vsyscall
.previous

And that's it.

Quoted 7 lines

What do you think? This comes with massively cleaned up kernel-side asm as well as a test case that actually validates the CFI directives.

Certainly, a bunch of your patches make sense regardless, and I'll review them and add them to my queue soon.

--Andy

How does the performance compare to the original? Looking at the disassembly, there are two added function calls, and it reloads the args from the stack instead of just shuffling registers.

Quoted 8 lines
Quoted 2 lines

The replacement is dramatically faster, which means I probably benchmarked it wrong. I'll try again in a day or two.

It's enough slower to be problematic. I need to figure out how to trace it properly. (Hmm? Maybe it's time to learn how to get perf on the host to trace a KVM guest.)

Everything is and was hilariously slow with context tracking on. That needs to get fixed, and hopefully once this entry stuff is done someone will do the other end of it.

I got random errors from perf kvm, but I think I found at least part of the issue. The two irqs_disabled() calls in common.c are kind of expensive. I should disable them on non-lockdep kernels.

The context tracking hooks are also too expensive, even when disabled. I should do something to optimize those. Hello, static keys? This doesn't affect syscalls, though.

With context tracking off and the irqs_disabled checks commented out, we're probably doing well enough. We can always tweak the C code and aggressively force inlining if we want a few cycles back.

--Andy

#14 lore raw
Quoted 11 lines · Andy Lutomirski

I got random errors from perf kvm, but I think I found at least part of the issue. The two irqs_disabled() calls in common.c are kind of expensive. I should disable them on non-lockdep kernels.

The context tracking hooks are also too expensive, even when disabled. I should do something to optimize those. Hello, static keys? This doesn't affect syscalls, though.

With context tracking off and the irqs_disabled checks commented out, we're probably doing well enough. We can always tweak the C code and aggressively force inlining if we want a few cycles back.

Currently, a compat AT_SYSINFO syscall (getpid) is 171 cycles for me. With my patches, it's 196 cycles, so it's really not that bad. The impact will probably be slightly worse on native 32-bit because of increased register pressure and because one of the micro-optimizations I threw in are 64-bit specific. We could probably tune the C code a bit more to get a few of the cycles back.

On the flip side, the rewrite is *far* faster in some of the slow path cases because the slow path no longer forces IRET.

On 32-bit, there's the added benefit that we could drop asmlinkage from the syscall bodies on top of the rewrite.

--Andy

Quoted 2 lines

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC