Skip to contentSkip to conversation

Why disable vdso by default with CONFIG_PARAVIRT?

DISCUSSION Jeremy Fitzhardinge 4 participants 17 messages Dec 12, 2006 – Dec 13, 2006 virtio core

lore
Share & tools
lore
#0 lore raw

Hi Andi,

What problem do they cause together? There's certainly no problem with Xen+vdso (in fact, its actually very useful so that it picks up the right libc with Xen-friendly TLS).

J

#1 lore raw
Quoted 6 lines · Jeremy Fitzhardinge

Hi Andi,

What problem do they cause together? There's certainly no problem with Xen+vdso (in fact, its actually very useful so that it picks up the right libc with Xen-friendly TLS).

Methinks the compat VDSO support got broken in the config? Paravirt + COMPAT_VDSO are incompatible.

#2 lore raw
Quoted 10 lines · Zachary Amsden
Quoted 6 lines · Jeremy Fitzhardinge

Hi Andi,

What problem do they cause together? There's certainly no problem with Xen+vdso (in fact, its actually very useful so that it picks up the right libc with Xen-friendly TLS).

Methinks the compat VDSO support got broken in the config? Paravirt + COMPAT_VDSO are incompatible.

Yes, that's true, but I'm looking at arch/i386/kernel/sysenter.c:

#ifdef CONFIG_PARAVIRT
unsigned int __read_mostly vdso_enabled = 0;
#else
unsigned int __read_mostly vdso_enabled = 1;
#endif

I can't think of any reason why that should be necessary.

J

#3 lore raw
Quoted 17 lines · Jeremy Fitzhardinge
Quoted 4 lines · Zachary Amsden
Quoted 2 lines · Jeremy Fitzhardinge

Hi Andi,

Quoted 8 lines
Quoted 5 lines

What problem do they cause together? There's certainly no problem with Xen+vdso (in fact, its actually very useful so that it picks up the right libc with Xen-friendly TLS).

Methinks the compat VDSO support got broken in the config? Paravirt + COMPAT_VDSO are incompatible.

Yes, that's true, but I'm looking at arch/i386/kernel/sysenter.c:

Quoted 8 lines

#ifdef CONFIG_PARAVIRT
unsigned int __read_mostly vdso_enabled = 0;
#else
unsigned int __read_mostly vdso_enabled = 1;
#endif

I can't think of any reason why that should be necessary.

It's not for us or Xen. Perhaps it came from lhype?

#4 lore raw
Quoted 1 line · Zachary Amsden

It's not for us or Xen. Perhaps it came from lhype?

(I suspect it came from Andi's fevered brain.) If lhype can't deal with vdso, it can turn it off for itself - but I don't think its a problem for lhype.

J

#5 lore raw
In reply to Jeremy Fitzhardinge
Quoted 2 lines · Jeremy Fitzhardinge

What problem do they cause together? There's certainly no problem with Xen+vdso

This was the change which finally got my test system (with an older SUSE 9.0 based user land to boot). With paravirt older glibc's ld.so otherwise throws assertation failures because it somehow can't deal with the new placement. This only happens with paravirt enabled.

Binary compatibility is important.

Quoted 2 lines

(in fact, its actually very useful so that it picks up the right libc with Xen-friendly TLS).

AFAIK libc selection comes from the aux vector, not the vdso.

-Andi

#6 lore raw
Quoted 6 lines · Andi Kleen
Quoted 3 lines · Jeremy Fitzhardinge

What problem do they cause together? There's certainly no problem with Xen+vdso

Quoted 7 lines

This was the change which finally got my test system (with an older SUSE 9.0 based user land to boot). With paravirt older glibc's ld.so otherwise throws assertation failures because it somehow can't deal with the new placement. This only happens with paravirt enabled.

Binary compatibility is important.

Yes, but the old placement of the vdso is incompatible with paravirt guests. The only solution I can think of to keep compatibility is to dynamically place the vdso during boot, but this is complex and introduces an indirection penalty to the fast sysenter syscall path (unless we make that a dynamic patch).

What should we do to fix this? Breaking compatibility for paravirt compilation is certainly the easiest thing to do.

Zach

#7 lore raw
In reply to Andi Kleen
Quoted 7 lines · Andi Kleen

This was the change which finally got my test system (with an older SUSE 9.0 based user land to boot). With paravirt older glibc's ld.so otherwise throws assertation failures because it somehow can't deal with the new placement. This only happens with paravirt enabled.

Binary compatibility is important.

So the SUSE 9 libc needs COMPAT_VDSO, or it just can't deal with the vdso moving around? Can it deal with the current kernel's mobile vdso?

Quoted 5 lines
Quoted 3 lines

(in fact, its actually very useful so that it picks up the right libc with Xen-friendly TLS).

AFAIK libc selection comes from the aux vector, not the vdso.

No, it seems to be done by adding a .note segment to the vdso share object. Without the vdso, my Xen test system doesn't boot because it can't find the nosegneg versions of the libraries (it doesn't seem to know where to look). I'm not sure how all this stuff fits together. But the auxv is supposed to point to the vdso...

J

#8 lore raw
Quoted 2 lines

So the SUSE 9

SL9.0 (not to be confused with SLES9). But it's all distributions with an older glibc.

Quoted 2 lines

libc needs COMPAT_VDSO, or it just can't deal with the
vdso moving around? Can it deal with the current kernel's mobile vdso?

It needs COMPAT_VDSO

[which is basically COMPAT_NO_OLD_GLIBC and imho always was a big mistake to have a config anyways -- one shouldn't gamble with binary compatibility so lightly]

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

(in fact, its actually very useful so that it picks up the right libc with Xen-friendly TLS).

AFAIK libc selection comes from the aux vector, not the vdso.

No, it seems to be done by adding a .note segment to the vdso share object.

Hmm, i had assumed it used the same mechanism as the CPU optimized libcs -- and that comes from the aux vector AT_PLATFORM.

Quoted 3 lines

Without the vdso, my Xen test system doesn't boot because it can't find the nosegneg versions of the libraries (it doesn't seem to know where to look). I'm not sure how all this stuff fits together.

Why does it not boot? At least in the past nosegneg was only a optimization to avoid some unnecessary traps to the hypervisor, but it should handle it. Has that changed?

Quoted 1 line

But the auxv is supposed to point to the vdso...

create_elf_tables() puts the AT_PLATFORM string onto the stack, unless i'm misreading the code badly. No dependency on vdso.

-Andi

#9 lore raw
Quoted 6 lines · Andi Kleen

It needs COMPAT_VDSO

[which is basically COMPAT_NO_OLD_GLIBC and imho always was a big mistake to have a config anyways -- one shouldn't gamble with binary compatibility so lightly]

It's unfortunate; there's a fundamental address space clash, and there's no nice resolution.

Will your system boot with vdso=0 on the kernel command line? Presumably it will boot paravirt-native without it (since native makes no claims on the address space), so its something you could put in your Xen config file, no?

Quoted 3 lines

Hmm, i had assumed it used the same mechanism as the CPU optimized libcs -- and that comes from the aux vector AT_PLATFORM.

There's some magic that involves a .note segment in the vdso itself, and a ld.so.conf entry which maps the string in there ("nosegneg") to a pseudo-hardware capability, and ld.so uses the result of that to look for more places for libraries. I don't really understand how all the pieces fit together.

Quoted 4 lines

Why does it not boot? At least in the past nosegneg was only a optimization to avoid some unnecessary traps to the hypervisor, but it should handle it. Has that changed?

No, but my (very stripped down) test system has no other libraries.

J

#10 lore raw
Quoted 1 line · Jeremy Fitzhardinge

Will your system boot with vdso=0 on the kernel command line?

Sure, it's the same as what I did by default.

Quoted 3 lines

Presumably it will boot paravirt-native without it (since native makes no claims on the address space), so its something you could put in your Xen config file, no?

I don't think being incompatible to old binaries is a sensible default. That is why I changed the wrong default. If paravirt ops cannot supply a compatible vdso it has to do without one.

-Andi

#11 lore raw
Quoted 3 lines · Andi Kleen

I don't think being incompatible to old binaries is a sensible default. That is why I changed the wrong default. If paravirt ops cannot supply a compatible vdso it has to do without one.

Do you know what glibc2.1 actually needs from the vdso? Does it actually interpret as an elf file, or just it just jump into it to perform syscalls? I wonder if we could use a fault in the vdso memory range to act as a syscall, or something like that?

J

#12 lore raw
Quoted 7 lines · Jeremy Fitzhardinge
Quoted 3 lines · Andi Kleen

I don't think being incompatible to old binaries is a sensible default. That is why I changed the wrong default. If paravirt ops cannot supply a compatible vdso it has to do without one.

Do you know what glibc2.1 actually needs from the vdso? Does it actually interpret as an elf file,

Interpret it as a ELF file, but then has some special hacks to jump directly anyways (or at least there is no direct linking, but it goes over a trampoline in the main glibc)

The failure is an assertation failure in ld.so.

-Andi

#13 lore raw
Quoted 6 lines · Andi Kleen
Quoted 5 lines · Jeremy Fitzhardinge
Quoted 3 lines · Andi Kleen

I don't think being incompatible to old binaries is a sensible default. That is why I changed the wrong default. If paravirt ops cannot supply a compatible vdso it has to do without one.

Quoted 8 lines
Quoted 2 lines

Do you know what glibc2.1 actually needs from the vdso? Does it actually interpret as an elf file,

Interpret it as a ELF file, but then has some special hacks to jump directly anyways (or at least there is no direct linking, but it goes over a trampoline in the main glibc)

The failure is an assertation failure in ld.so.

And since init is special, the SIGABRT doesn't get delivered. Andi, does this hack come close? (Against older kernel, you'll need to take out the #ifdef CONFIG_PARAVIRT around the vdso_enabled initialization).

Rusty.

Older glibcs assert() that the vdso will be in a particular spot (which it can no longer be with CONFIG_PARAVIRT). As this glibc was shipped in SuSE 9.0 and Fedora Core 1, it's not a trivial breakage.

Try to detect the failing init at runtime, turn off vdso and re-exec. Untested, since I don't have a failing system.

arch/i386/Kconfig +8 -9
diff -r c3d6f0e043e0 arch/i386/Kconfig
@@ -848,15 +848,14 @@ config HOTPLUG_CPU
 	  /sys/devices/system/cpu.
 
 config COMPAT_VDSO
-	bool "Compat VDSO support"
-	default y
-	depends on !PARAVIRT
-	help
-	  Map the VDSO to the predictable old-style address too.
-	---help---
-	  Say N here if you are running a sufficiently recent glibc
-	  version (2.3.3 or later), to remove the high-mapped
-	  VDSO mapping and to exclusively use the randomized VDSO.
+	bool "Disable VDSO for old glibc"
+	default y
+	---help---
+	  Old glibc does not like the modern VDSO placement (glibc
+	  2.3.3 or later is fine, Fedora Core 1 and SuSE 9.0 have
+	  problems).  Very old glibc versions don't use the VDSO at
+	  all.  This option tries to detect the glibc assertion which
+	  occurs and then disables the VDSO.
 
 	  If unsure, say Y.
 
arch/i386/kernel/signal.c +11 -0
diff -r c3d6f0e043e0 arch/i386/kernel/signal.c
@@ -608,6 +608,17 @@ static void fastcall do_signal(struct pt
 
 		return;
 	}
+#ifdef CONFIG_COMPAT_VDSO
+	else if (signr == -1) {
+		void reexec_init(void);
+		static int reexec_done;
+		if (!reexec_done++) {
+			printk("COMPAT_VDSO: old glibc?  Disabling vdso\n");
+			vdso_enabled = 0;
+			reexec_init();
+		}
+	}
+#endif
 
 	/* Did we come from a system call? */
 	if (regs->orig_eax >= 0) {
arch/i386/kernel/sysenter.c +0 -5
diff -r c3d6f0e043e0 arch/i386/kernel/sysenter.c
@@ -72,15 +72,10 @@ int __init sysenter_setup(void)
 {
 	syscall_page = (void *)get_zeroed_page(GFP_ATOMIC);
 
-#ifdef CONFIG_COMPAT_VDSO
-	__set_fixmap(FIX_VDSO, __pa(syscall_page), PAGE_READONLY);
-	printk("Compat vDSO mapped to %08lx.\n", __fix_to_virt(FIX_VDSO));
-#else
 	/*
 	 * In the non-compat case the ELF coredumping code needs the fixmap:
 	 */
 	__set_fixmap(FIX_VDSO, __pa(syscall_page), PAGE_KERNEL_RO);
-#endif
 
 	if (!boot_cpu_has(X86_FEATURE_SEP)) {
 		memcpy(syscall_page,
arch/i386/mm/pgtable.c +0 -6
diff -r c3d6f0e043e0 arch/i386/mm/pgtable.c
@@ -141,10 +141,8 @@ void set_pmd_pfn(unsigned long vaddr, un
 }
 
 static int fixmaps;
-#ifndef CONFIG_COMPAT_VDSO
 unsigned long __FIXADDR_TOP = 0xfffff000;
 EXPORT_SYMBOL(__FIXADDR_TOP);
-#endif
 
 void __set_fixmap (enum fixed_addresses idx, unsigned long phys, pgprot_t flags)
 {
@@ -168,12 +166,8 @@ void reserve_top_address(unsigned long r
 void reserve_top_address(unsigned long reserve)
 {
 	BUG_ON(fixmaps > 0);
-#ifdef CONFIG_COMPAT_VDSO
-	BUG_ON(reserve != 0);
-#else
 	__FIXADDR_TOP = -reserve - PAGE_SIZE;
 	__VMALLOC_RESERVE += reserve;
-#endif
 }
 
 pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
include/asm-i386/elf.h +0 -5
diff -r c3d6f0e043e0 include/asm-i386/elf.h
@@ -135,13 +135,8 @@ extern int dump_task_extended_fpu (struc
 #define VDSO_HIGH_BASE		(__fix_to_virt(FIX_VDSO))
 #define VDSO_BASE		((unsigned long)current->mm->context.vdso)
 
-#ifdef CONFIG_COMPAT_VDSO
-# define VDSO_COMPAT_BASE	VDSO_HIGH_BASE
-# define VDSO_PRELINK		VDSO_HIGH_BASE
-#else
 # define VDSO_COMPAT_BASE	VDSO_BASE
 # define VDSO_PRELINK		0
-#endif
 
 #define VDSO_COMPAT_SYM(x) \
 		(VDSO_COMPAT_BASE + (unsigned long)(x) - VDSO_PRELINK)
include/asm-i386/fixmap.h +0 -4
diff -r c3d6f0e043e0 include/asm-i386/fixmap.h
@@ -19,11 +19,7 @@
  * Leave one empty page between vmalloc'ed areas and
  * the start of the fixmap.
  */
-#ifndef CONFIG_COMPAT_VDSO
 extern unsigned long __FIXADDR_TOP;
-#else
-#define __FIXADDR_TOP  0xfffff000
-#endif
 
 #ifndef __ASSEMBLY__
 #include <linux/kernel.h>
init/main.c +7 -0
diff -r c3d6f0e043e0 init/main.c
@@ -707,6 +707,13 @@ static void run_init_process(char *init_
 	kernel_execve(init_filename, argv_init, envp_init);
 }
 
+#ifdef CONFIG_COMPAT_VDSO
+void reexec_init(void)
+{
+	kernel_execve(argv_init[0], argv_init, envp_init);
+}
+#endif
+
 static int init(void * unused)
 {
 	lock_kernel();
kernel/signal.c +10 -1
diff -r c3d6f0e043e0 kernel/signal.c
@@ -2010,8 +2010,17 @@ relock:
 		 * within that pid space. It can of course get signals from
 		 * its parent pid space.
 		 */
-		if (current == child_reaper(current))
+		if (current == child_reaper(current)) {
+#ifdef CONFIG_COMPAT_VDSO
+			/* Gross hack: Old glibc asserts, not
+			   liking moved vdso (SuSE 9, FC1) */
+			if (signr == SIGABRT && list_empty(&current->children)) {
+				signr = -1;
+				break;
+			}
+#endif				
 			continue;
+		}
 
 		if (sig_kernel_stop(signr)) {
 			/*
#14 lore raw
In reply to Andi Kleen
Quoted 1 line · Andi Kleen

The failure is an assertation failure in ld.so.

OK, this patch tested on an assert() in init.

===
Older glibcs assert() that the vdso will be in a particular spot (which it can no longer be with CONFIG_PARAVIRT). As this glibc was shipped in SuSE 9.0 and Fedora Core 1, it's not a trivial breakage.

Try to detect the failing init at runtime, turn off vdso and re-exec. Untested on the actual failing systems, but should work.

arch/i386/Kconfig +8 -9
diff -r ed1ffbd17965 arch/i386/Kconfig
@@ -816,15 +816,14 @@ config HOTPLUG_CPU
 	  /sys/devices/system/cpu.
 
 config COMPAT_VDSO
-	bool "Compat VDSO support"
-	default y
-	depends on !PARAVIRT
-	help
-	  Map the VDSO to the predictable old-style address too.
-	---help---
-	  Say N here if you are running a sufficiently recent glibc
-	  version (2.3.3 or later), to remove the high-mapped
-	  VDSO mapping and to exclusively use the randomized VDSO.
+	bool "Disable VDSO for old glibc"
+	default y
+	---help---
+	  Old glibc does not like the modern VDSO placement (glibc
+	  2.3.3 or later is fine, Fedora Core 1 and SuSE 9.0 have
+	  problems).  Very old glibc versions don't use the VDSO at
+	  all.  This option tries to detect the glibc assertion which
+	  occurs and then disables the VDSO.
 
 	  If unsure, say Y.
 
arch/i386/kernel/signal.c +14 -0
diff -r ed1ffbd17965 arch/i386/kernel/signal.c
@@ -608,6 +608,20 @@ static void fastcall do_signal(struct pt
 
 		return;
 	}
+#ifdef CONFIG_COMPAT_VDSO
+	else if (signr == -1) {
+		void reexec_init(void);
+		if (vdso_enabled) {
+			printk(KERN_WARNING "COMPAT_VDSO: Old glibc?"
+			       " Re-execing init with vdso disabled\n");
+			vdso_enabled = 0;
+			/* kill() made us think we're dying: we're not. */
+			current->signal->group_stop_count = 0;
+			reexec_init();
+			printk(KERN_WARNING "Re-exec of init failed\n");
+		}
+	}
+#endif
 
 	/* Did we come from a system call? */
 	if (regs->orig_eax >= 0) {
arch/i386/kernel/sysenter.c +0 -9
diff -r ed1ffbd17965 arch/i386/kernel/sysenter.c
@@ -27,11 +27,7 @@
  * Should the kernel map a VDSO page into processes and pass its
  * address down to glibc upon exec()?
  */
-#ifdef CONFIG_PARAVIRT
-unsigned int __read_mostly vdso_enabled = 0;
-#else
 unsigned int __read_mostly vdso_enabled = 1;
-#endif
 
 EXPORT_SYMBOL_GPL(vdso_enabled);
 
@@ -76,15 +72,10 @@ int __init sysenter_setup(void)
 {
 	syscall_page = (void *)get_zeroed_page(GFP_ATOMIC);
 
-#ifdef CONFIG_COMPAT_VDSO
-	__set_fixmap(FIX_VDSO, __pa(syscall_page), PAGE_READONLY);
-	printk("Compat vDSO mapped to %08lx.\n", __fix_to_virt(FIX_VDSO));
-#else
 	/*
 	 * In the non-compat case the ELF coredumping code needs the fixmap:
 	 */
 	__set_fixmap(FIX_VDSO, __pa(syscall_page), PAGE_KERNEL_RO);
-#endif
 
 	if (!boot_cpu_has(X86_FEATURE_SEP)) {
 		memcpy(syscall_page,
arch/i386/mm/pgtable.c +0 -6
diff -r ed1ffbd17965 arch/i386/mm/pgtable.c
@@ -144,10 +144,8 @@ void set_pmd_pfn(unsigned long vaddr, un
 }
 
 static int fixmaps;
-#ifndef CONFIG_COMPAT_VDSO
 unsigned long __FIXADDR_TOP = 0xfffff000;
 EXPORT_SYMBOL(__FIXADDR_TOP);
-#endif
 
 void __set_fixmap (enum fixed_addresses idx, unsigned long phys, pgprot_t flags)
 {
@@ -171,12 +169,8 @@ void reserve_top_address(unsigned long r
 void reserve_top_address(unsigned long reserve)
 {
 	BUG_ON(fixmaps > 0);
-#ifdef CONFIG_COMPAT_VDSO
-	BUG_ON(reserve != 0);
-#else
 	__FIXADDR_TOP = -reserve - PAGE_SIZE;
 	__VMALLOC_RESERVE += reserve;
-#endif
 }
 
 pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
include/asm-i386/elf.h +0 -5
diff -r ed1ffbd17965 include/asm-i386/elf.h
@@ -135,13 +135,8 @@ extern int dump_task_extended_fpu (struc
 #define VDSO_HIGH_BASE		(__fix_to_virt(FIX_VDSO))
 #define VDSO_BASE		((unsigned long)current->mm->context.vdso)
 
-#ifdef CONFIG_COMPAT_VDSO
-# define VDSO_COMPAT_BASE	VDSO_HIGH_BASE
-# define VDSO_PRELINK		VDSO_HIGH_BASE
-#else
 # define VDSO_COMPAT_BASE	VDSO_BASE
 # define VDSO_PRELINK		0
-#endif
 
 #define VDSO_COMPAT_SYM(x) \
 		(VDSO_COMPAT_BASE + (unsigned long)(x) - VDSO_PRELINK)
include/asm-i386/fixmap.h +0 -4
diff -r ed1ffbd17965 include/asm-i386/fixmap.h
@@ -19,11 +19,7 @@
  * Leave one empty page between vmalloc'ed areas and
  * the start of the fixmap.
  */
-#ifndef CONFIG_COMPAT_VDSO
 extern unsigned long __FIXADDR_TOP;
-#else
-#define __FIXADDR_TOP  0xfffff000
-#endif
 
 #ifndef __ASSEMBLY__
 #include <linux/kernel.h>
init/main.c +10 -0
diff -r ed1ffbd17965 init/main.c
@@ -710,6 +710,16 @@ static void run_init_process(char *init_
 	kernel_execve(init_filename, argv_init, envp_init);
 }
 
+#ifdef CONFIG_COMPAT_VDSO
+void reexec_init(void)
+{
+	mm_segment_t oldfs = get_fs();
+	set_fs(KERNEL_DS);
+	kernel_execve(argv_init[0], argv_init, envp_init);
+	set_fs(oldfs);
+}
+#endif
+
 static int init(void * unused)
 {
 	lock_kernel();
kernel/signal.c +10 -1
diff -r ed1ffbd17965 kernel/signal.c
@@ -1878,8 +1878,17 @@ relock:
 			continue;
 
 		/* Init gets no signals it doesn't want.  */
-		if (current == child_reaper)
+		if (current == child_reaper) {
+#ifdef CONFIG_COMPAT_VDSO
+			/* Gross hack: Old glibc asserts, not
+			   liking moved vdso (SuSE 9, FC1) */
+			if (signr == SIGABRT) {
+				signr = -1;
+				break;
+			}
+#endif				
 			continue;
+		}
 
 		if (sig_kernel_stop(signr)) {
 			/*
#15 lore raw
Quoted 4 lines · Rusty Russell
Quoted 1 line · Andi Kleen

The failure is an assertation failure in ld.so.

OK, this patch tested on an assert() in init.

And this variant traps kill instead which is simpler:

Older glibcs assert() that the vdso will be in a particular spot (which it can no longer be with CONFIG_PARAVIRT). As this glibc was shipped in SuSE 9.0 and Fedora Core 1, it's not a trivial breakage.

Try to detect the failing init at runtime, turn off vdso and re-exec. Not tested on this particular assertion, but should work.

arch/i386/Kconfig +8 -9
diff -r 2d9ddfd41f3a arch/i386/Kconfig
@@ -816,15 +816,14 @@ config HOTPLUG_CPU
 	  /sys/devices/system/cpu.
 
 config COMPAT_VDSO
-	bool "Compat VDSO support"
-	default y
-	depends on !PARAVIRT
-	help
-	  Map the VDSO to the predictable old-style address too.
-	---help---
-	  Say N here if you are running a sufficiently recent glibc
-	  version (2.3.3 or later), to remove the high-mapped
-	  VDSO mapping and to exclusively use the randomized VDSO.
+	bool "Disable VDSO for old glibc"
+	default y
+	---help---
+	  Old glibc does not like the modern VDSO placement (glibc
+	  2.3.3 or later is fine, Fedora Core 1 and SuSE 9.0 have
+	  problems).  Very old glibc versions don't use the VDSO at
+	  all.  This option tries to detect the glibc assertion which
+	  occurs and then disables the VDSO.
 
 	  If unsure, say Y.
 
arch/i386/kernel/signal.c +19 -0
diff -r 2d9ddfd41f3a arch/i386/kernel/signal.c
@@ -655,3 +655,22 @@ void do_notify_resume(struct pt_regs *re
 	
 	clear_thread_flag(TIF_IRET);
 }
+
+#ifdef CONFIG_COMPAT_VDSO
+#include <linux/syscalls.h>
+
+asmlinkage long
+sys_check_init_abort_kill(int pid, int sig)
+{
+	if (unlikely(current == child_reaper)
+	    && pid == 1 && sig == SIGABRT && vdso_enabled) {
+		void reexec_init(void);
+		printk(KERN_WARNING "COMPAT_VDSO: Old glibc?"
+		       " Re-execing init with vdso disabled\n");
+		vdso_enabled = 0;
+		reexec_init();
+		printk(KERN_WARNING "Re-exec of init failed\n");
+	}
+	return sys_kill(pid, sig);
+}
+#endif
arch/i386/kernel/syscall_table.S +4 -0
diff -r 2d9ddfd41f3a arch/i386/kernel/syscall_table.S
@@ -36,7 +36,11 @@ ENTRY(sys_call_table)
 	.long sys_nice
 	.long sys_ni_syscall	/* 35 - old ftime syscall holder */
 	.long sys_sync
+#ifdef CONFIG_COMPAT_VDSO
+	.long sys_check_init_abort_kill
+#else
 	.long sys_kill
+#endif
 	.long sys_rename
 	.long sys_mkdir
 	.long sys_rmdir		/* 40 */
arch/i386/kernel/sysenter.c +0 -9
diff -r 2d9ddfd41f3a arch/i386/kernel/sysenter.c
@@ -27,11 +27,7 @@
  * Should the kernel map a VDSO page into processes and pass its
  * address down to glibc upon exec()?
  */
-#ifdef CONFIG_PARAVIRT
-unsigned int __read_mostly vdso_enabled = 0;
-#else
 unsigned int __read_mostly vdso_enabled = 1;
-#endif
 
 EXPORT_SYMBOL_GPL(vdso_enabled);
 
@@ -76,15 +72,10 @@ int __init sysenter_setup(void)
 {
 	syscall_page = (void *)get_zeroed_page(GFP_ATOMIC);
 
-#ifdef CONFIG_COMPAT_VDSO
-	__set_fixmap(FIX_VDSO, __pa(syscall_page), PAGE_READONLY);
-	printk("Compat vDSO mapped to %08lx.\n", __fix_to_virt(FIX_VDSO));
-#else
 	/*
 	 * In the non-compat case the ELF coredumping code needs the fixmap:
 	 */
 	__set_fixmap(FIX_VDSO, __pa(syscall_page), PAGE_KERNEL_RO);
-#endif
 
 	if (!boot_cpu_has(X86_FEATURE_SEP)) {
 		memcpy(syscall_page,
arch/i386/mm/pgtable.c +0 -6
diff -r 2d9ddfd41f3a arch/i386/mm/pgtable.c
@@ -144,10 +144,8 @@ void set_pmd_pfn(unsigned long vaddr, un
 }
 
 static int fixmaps;
-#ifndef CONFIG_COMPAT_VDSO
 unsigned long __FIXADDR_TOP = 0xfffff000;
 EXPORT_SYMBOL(__FIXADDR_TOP);
-#endif
 
 void __set_fixmap (enum fixed_addresses idx, unsigned long phys, pgprot_t flags)
 {
@@ -171,12 +169,8 @@ void reserve_top_address(unsigned long r
 void reserve_top_address(unsigned long reserve)
 {
 	BUG_ON(fixmaps > 0);
-#ifdef CONFIG_COMPAT_VDSO
-	BUG_ON(reserve != 0);
-#else
 	__FIXADDR_TOP = -reserve - PAGE_SIZE;
 	__VMALLOC_RESERVE += reserve;
-#endif
 }
 
 pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
include/asm-i386/elf.h +0 -5
diff -r 2d9ddfd41f3a include/asm-i386/elf.h
@@ -135,13 +135,8 @@ extern int dump_task_extended_fpu (struc
 #define VDSO_HIGH_BASE		(__fix_to_virt(FIX_VDSO))
 #define VDSO_BASE		((unsigned long)current->mm->context.vdso)
 
-#ifdef CONFIG_COMPAT_VDSO
-# define VDSO_COMPAT_BASE	VDSO_HIGH_BASE
-# define VDSO_PRELINK		VDSO_HIGH_BASE
-#else
 # define VDSO_COMPAT_BASE	VDSO_BASE
 # define VDSO_PRELINK		0
-#endif
 
 #define VDSO_COMPAT_SYM(x) \
 		(VDSO_COMPAT_BASE + (unsigned long)(x) - VDSO_PRELINK)
include/asm-i386/fixmap.h +0 -4
diff -r 2d9ddfd41f3a include/asm-i386/fixmap.h
@@ -19,11 +19,7 @@
  * Leave one empty page between vmalloc'ed areas and
  * the start of the fixmap.
  */
-#ifndef CONFIG_COMPAT_VDSO
 extern unsigned long __FIXADDR_TOP;
-#else
-#define __FIXADDR_TOP  0xfffff000
-#endif
 
 #ifndef __ASSEMBLY__
 #include <linux/kernel.h>
init/main.c +10 -0
diff -r 2d9ddfd41f3a init/main.c
@@ -710,6 +710,16 @@ static void run_init_process(char *init_
 	kernel_execve(init_filename, argv_init, envp_init);
 }
 
+#ifdef CONFIG_COMPAT_VDSO
+void reexec_init(void)
+{
+	mm_segment_t oldfs = get_fs();
+	set_fs(KERNEL_DS);
+	kernel_execve(argv_init[0], argv_init, envp_init);
+	set_fs(oldfs);
+}
+#endif
+
 static int init(void * unused)
 {
 	lock_kernel();
#16 lore raw
In reply to Andi Kleen
Quoted 10 lines · Andi Kleen
Quoted 2 lines · Jeremy Fitzhardinge

What problem do they cause together? There's certainly no problem with Xen+vdso

This was the change which finally got my test system (with an older SUSE 9.0 based user land to boot). With paravirt older glibc's ld.so otherwise throws assertation failures because it somehow can't deal with the new placement. This only happens with paravirt enabled.

Binary compatibility is important.

Yes, this goes back to the original COMPAT_VDSO config option, months ago. FC1's buggy glibc couldn't handle the vdso being in an unusual place, and (over my objections) the COMPAT_VDSO option was introduced. Seems like SuSE 9.0 is similarly effected.

I don't have a system which has this problem (and, at the end of a modem, I'm unlikely to get one soon). But I would suggest that COMPAT_VDSO should be rewritten: catch the segv from init (presumably in around the old vdso 0xFFFF0000 addr?), printk a message, turn vdso off and re-exec init.

That should make everyone happy. Now, someone who can repro this please code it up!

Thanks!
Rusty.