x86-64 vDSO changes for 3.1
Subsystems (7)
v2 cover x86-64 vDSO changes for 3.1
This series applies to the x86/vdso branch of the -tip tree.
The first patch cleans up the vsyscall emulation code.
After the vsyscall emulation patches, the only real executable code left in the vsyscall page is vread_tsc and vread_hpet. That code is only called from the vDSO, so patches 2-6 move it into the vDSO.
vread_tsc() uses rdtsc_barrier(), which contains two alternatives. Patches 2 and 3 make alternative patching work in the vDSO. (This has a slightly odd side effect that the vDSO image dumped from memory doesn't quite match the debug version anymore, but it's hard to imagine that causing problems.)
Patch 4 fixes an annoyance I found while writing this code. If you introduce an undefined symbol into the vDSO, you get an unhelpful error message. ld is smart enough to give a nice error if you ask it to.
Patch 5 cleans up the architecture-specific part of struct clocksource. IA64 had its own ifdefed code in there, and the supposedly generic vread pointer was only used by x86-64. With the patch, each arch gets to set up its own private part of struct clocksource.
Patch 6 is the meat. It moves vread_tsc and vread_hpet into the vDSO where they belong, and it's a net deletion of code because it removes a bunch of magic needed to make regular functions accessible through the vsyscall page.
With patches 1-6 applied, every single byte in the vsyscall page is some sort of trap instruction.
Patches 7 and 8 are optional. Patch 7 changes IA64 to use the new arch gtod data. It presumably should not go in through the x86 tree. Patch 8 adds some vDSO documentation and a reference vDSO parser for user code to use. It's meant for projects that don't dynamically link to glibc (e.g. Go) but still want to call the vDSO. Someone who knows more about ELF than I should take a look.
*** Note to IA64 people: I have not even compile-tested this on IA64. ***
Changes from v1:
- Tidy up vDSO alternative patching (thanks, Borislav).
- Fix really dumb bugs in the IA64 stuff.
- Add the cleanup patch and the reference vDSO parser.
- Split the main IA-64 patch out.
Andy Lutomirski (8):
x86-64: Improve vsyscall emulation CS and RIP handling
x86: Make alternative instruction pointers relative
x86-64: Allow alternative patching in the vDSO
x86-64: Add --no-undefined to vDSO build
clocksource: Replace vread with generic arch data
x86-64: Move vread_tsc and vread_hpet into the vDSO
ia64: Replace clocksource.fsys_mmio with generic arch data Document the vDSO and add a reference parser
Documentation/ABI/stable/vdso | 27 ++++ Documentation/vDSO/parse_vdso.c | 256 ++++++++++++++++++++++++++++++++ Documentation/vDSO/vdso_test.c | 112 ++++++++++++++ arch/ia64/include/asm/clocksource.h | 12 ++ arch/ia64/kernel/cyclone.c | 2 +- arch/ia64/kernel/time.c | 2 +- arch/ia64/sn/kernel/sn2/timer.c | 2 +- arch/x86/include/asm/alternative-asm.h | 4 +- arch/x86/include/asm/alternative.h | 8 +- arch/x86/include/asm/clocksource.h | 20 +++ arch/x86/include/asm/cpufeature.h | 8 +- arch/x86/include/asm/tsc.h | 4 - arch/x86/include/asm/vgtod.h | 2 +- arch/x86/include/asm/vsyscall.h | 16 -- arch/x86/kernel/Makefile | 7 +- arch/x86/kernel/alternative.c | 23 ++-- arch/x86/kernel/hpet.c | 9 +- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 3 - arch/x86/kernel/vread_tsc_64.c | 36 ----- arch/x86/kernel/vsyscall_64.c | 61 +++++--- arch/x86/lib/copy_page_64.S | 9 +- arch/x86/lib/memmove_64.S | 11 +- arch/x86/vdso/Makefile | 1 + arch/x86/vdso/vclock_gettime.c | 52 ++++++- arch/x86/vdso/vma.c | 30 ++++ drivers/char/hpet.c | 2 +- include/asm-generic/clocksource.h | 4 + include/linux/clocksource.h | 13 +- 29 files changed, 589 insertions(+), 149 deletions(-) create mode 100644 Documentation/ABI/stable/vdso create mode 100644 Documentation/vDSO/parse_vdso.c create mode 100644 Documentation/vDSO/vdso_test.c create mode 100644 arch/ia64/include/asm/clocksource.h create mode 100644 arch/x86/include/asm/clocksource.h delete mode 100644 arch/x86/kernel/vread_tsc_64.c create mode 100644 include/asm-generic/clocksource.h
Signature · 1 line
1.7.6
PATCH v2 1/8 x86-64: Improve vsyscall emulation CS and RIP handling
Two fixes here:
- Send SIGSEGV if called from compat code or with a funny CS.
- Don't BUG on impossible addresses.
This patch also removes an unused variable.
--- arch/x86/include/asm/vsyscall.h | 12 -------- arch/x86/kernel/vsyscall_64.c | 59 +++++++++++++++++++++++++-------------- 2 files changed, 38 insertions(+), 33 deletions(-)
arch/x86/include/asm/vsyscall.h +0 -12
@@ -31,18 +31,6 @@ extern struct timezone sys_tz;extern void map_vsyscall (void ); -/* Emulation */ - -static inline bool is_vsyscall_entry (unsigned long addr) -{ -return (addr & ~0xC00UL ) ==VSYSCALL_START ; -} - -static inline int vsyscall_entry_nr (unsigned long addr) -{ -return (addr &0xC00UL ) >>10 ; -} -#endif /* __KERNEL__ */ #endif /* _ASM_X86_VSYSCALL_H */
arch/x86/kernel/vsyscall_64.c +38 -21
108 lines not on this page · raw message
Signature · 1 line
1.7.6
Borislav Petkov
maintainerRe: [PATCH v2 1/8] x86-64: Improve vsyscall emulation CS and RIP handling
Quoted 60 lines · Andy Lutomirski
Two fixes here:
- Send SIGSEGV if called from compat code or with a funny CS.
- Don't BUG on impossible addresses.This patch also removes an unused variable.
Signed-off-by Andy Lutomirski <[email protected]>--- arch/x86/include/asm/vsyscall.h | 12 -------- arch/x86/kernel/vsyscall_64.c | 59 +++++++++++++++++++++++++-------------- 2 files changed, 38 insertions(+), 33 deletions(-)arch/x86/include/asm/vsyscall.h +0
-12@@ -31,18 +31,6 @@ extern struct timezone sys_tz;extern void map_vsyscall (void ); -/* Emulation */ - -static inline bool is_vsyscall_entry (unsigned long addr) -{ -return (addr & ~0xC00UL ) ==VSYSCALL_START ; -} - -static inline int vsyscall_entry_nr (unsigned long addr) -{ -return (addr &0xC00UL ) >>10 ; -} -#endif /* __KERNEL__ */ #endif /* _ASM_X86_VSYSCALL_H */ arch/x86/kernel/vsyscall_64.c +7
-2@@ -38,6 +38,7 @@#include <asm/vsyscall.h> #include <asm/pgtable.h> +#include <asm/compat.h> #include <asm/page.h> #include <asm/unistd.h> #include <asm/fixmap.h> @@ -97,33 +98,60 @@ static void warn_bad_vsyscall(const char *level, struct pt_regs *regs, tsk = current; -printk ("%s%s[%d] %s ip:%lx sp:%lx ax:%lx si:%lx di:%lx\n" , +printk ("%s%s[%d] %s ip:%lx cs:%lx sp:%lx ax:%lx si:%lx di:%lx\n" , level, tsk->comm,task_pid_nr (tsk), - message, regs->ip -2 , regs->sp, regs->ax, regs->si, regs->di); + message, regs->ip -2 , regs->cs, + regs->sp, regs->ax, regs->si, regs->di); +} + +static int addr_to_vsyscall_nr (unsigned *vsyscall_nr,unsigned long addr)
arch/x86/kernel/vsyscall_64.c static void warn_bad_vsyscall(const char *level, struct pt_regs *regs, · line 117
Quoted 10 lines
arch/x86/kernel/vsyscall_64.c +9
-0+{ +if ((addr & ~0xC00UL ) !=VSYSCALL_START ) +return -EINVAL ; + + *vsyscall_nr = (addr &0xC00UL ) >>10 ; +if (*vsyscall_nr >=3 ) +return -EINVAL ; + +return 0 ; }
I'm wondering: why don't you make this function return negative value on error, i.e. -EINVAL and the vsyscall number on success so that you can get rid of returning it through the arg pointer?
Then at the callsite you can do:
vsyscall_nr = addr_to_vsyscall_nr(addr); if (vsyscall_nr < 0) warn_bad_vsyscall(...)
?
Quoted 82 lines
arch/x86/kernel/vsyscall_64.c +22
-19void dotraplinkage do_emulate_vsyscall(struct pt_regs *regs, long error_code) { -const char *vsyscall_name;struct task_struct *tsk;unsigned long caller; -int vsyscall_nr; +unsigned vsyscall_nr;long ret; -/* Kernel code must never get here. */ -BUG_ON (!user_mode (regs)); -local_irq_enable ();/* +* Real 64-bit user mode code has cs == __USER_CS. Anything else +* is bogus. +*/ +if (regs->cs != __USER_CS) { +/* +* If we trapped from kernel mode, we might as well OOPS now +* instead of returning to some random address and OOPSing +* then. +*/ +BUG_ON (!user_mode (regs)); + +/* Compat mode and non-compat 32-bit CS should both segfault. */ +warn_bad_vsyscall (KERN_WARNING , regs, +"illegal int 0xcc from 32-bit mode" ); + goto sigsegv; + } + +/* * x86-ism here: regs->ip points to the instruction after the int 0xcc, * and int 0xcc is two bytes long. */ -if (!is_vsyscall_entry (regs->ip -2 )) { -warn_bad_vsyscall (KERN_WARNING , regs,"illegal int 0xcc (exploit attempt?)" ); +if (addr_to_vsyscall_nr (&vsyscall_nr, regs->ip -2 ) !=0 ) { +warn_bad_vsyscall (KERN_WARNING , regs, +"illegal int 0xcc (exploit attempt?)" ); goto sigsegv; } - vsyscall_nr =vsyscall_entry_nr (regs->ip -2 );if (get_user (caller, (unsigned long __user *)regs->sp) !=0 ) {warn_bad_vsyscall (KERN_WARNING , regs,"int 0xcc with bad stack (exploit attempt?)" );@@ -136,31 +164,20 @@ void dotraplinkage do_emulate_vsyscall(struct pt_regs *regs, long error_code)switch (vsyscall_nr) {case 0 : - vsyscall_name ="gettimeofday" ; ret =sys_gettimeofday ( (struct timeval __user *)regs->di, (struct timezone __user *)regs->si);break ;case 1 : - vsyscall_name ="time" ; ret =sys_time ((time_t __user *)regs->di);break ;case 2 : - vsyscall_name ="getcpu" ; ret =sys_getcpu ((unsigned __user *)regs->di, (unsigned __user *)regs->si,0 );break ; - -default : -/* -* If we get here, then vsyscall_nr indicates that int 0xcc -* happened at an address in the vsyscall page that doesn't -* contain int 0xcc. That can't happen. -*/ -BUG (); }if (ret == -EFAULT ) {Signature · 1 line
1.7.6
Thanks.
Signature · 2 lines
Regards/Gruss,
Boris.Re: [PATCH v2 1/8] x86-64: Improve vsyscall emulation CS and RIP handling
Quoted 76 lines · Borislav Petkov
Quoted 60 lines · Andy Lutomirski
Two fixes here:
- Send SIGSEGV if called from compat code or with a funny CS.
- Don't BUG on impossible addresses.This patch also removes an unused variable.
Signed-off-by Andy Lutomirski <[email protected]>---
arch/x86/include/asm/vsyscall.h | 12 --------arch/x86/kernel/vsyscall_64.c| 59 +++++++++++++++++++++++++--------------2 files changed, 38 insertions(+), 33 deletions(-)arch/x86/include/asm/vsyscall.h +0
-12@@ -31,18 +31,6 @@ extern struct timezone sys_tz; extern void map_vsyscall(void); -/* Emulation */ - -static inline bool is_vsyscall_entry (unsigned long addr) -{ - return (addr & ~0xC00UL) == VSYSCALL_START; -} - -static inline int vsyscall_entry_nr (unsigned long addr) -{ - return (addr & 0xC00UL) >> 10; -} - #endif /* __KERNEL__ */ #endif /* _ASM_X86_VSYSCALL_H */arch/x86/kernel/vsyscall_64.c +7
-2@@ -38,6 +38,7 @@ #include <asm/vsyscall.h> #include <asm/pgtable.h> +#include <asm/compat.h> #include <asm/page.h> #include <asm/unistd.h> #include <asm/fixmap.h>@@ -97,33 +98,60 @@ static void warn_bad_vsyscall(const char *level, struct pt_regs *regs, tsk = current; - printk("%s%s[%d] %s ip:%lx sp:%lx ax:%lx si:%lx di:%lx\n", + printk("%s%s[%d] %s ip:%lx cs:%lx sp:%lx ax:%lx si:%lx di:%lx\n", level, tsk->comm, task_pid_nr(tsk), - message, regs->ip - 2, regs->sp, regs->ax, regs->si, regs->di); + message, regs->ip - 2, regs->cs, + regs->sp, regs->ax, regs->si, regs->di); +} + +static int addr_to_vsyscall_nr (unsigned *vsyscall_nr,unsigned long addr)arch/x86/kernel/vsyscall_64.c static void warn_bad_vsyscall(const char *level, struct pt_regs *regs, · line 117
Quoted 10 lines
arch/x86/kernel/vsyscall_64.c +9
-0+{ + if ((addr & ~0xC00UL) != VSYSCALL_START) + return -EINVAL; + + *vsyscall_nr = (addr & 0xC00UL) >> 10; + if (*vsyscall_nr >= 3) + return -EINVAL; + + return 0; }I'm wondering: why don't you make this function return negative value on error, i.e.
-EINVALand the vsyscall number on success so that you can get rid of returning it through the arg pointer?
Quoted 5 lines
Then at the callsite you can do:
vsyscall_nr =
addr_to_vsyscall_nr(addr);
if (vsyscall_nr< 0)
warn_bad_vsyscall(...)
Because I don't want a warning about ret being used without being initialized.
With the code in this patch, the compiler is smart enough to figure out that either vsyscall_nr is 0, 1, or 2 or that the EINVAL branch is taken. I'll see if it works the other way.
--Andy
Borislav Petkov
maintainerRe: [PATCH v2 1/8] x86-64: Improve vsyscall emulation CS and RIP handling
Quoted 11 lines · Andrew Lutomirski
Quoted 9 lines
I'm wondering: why don't you make this function return negative value on error, i.e.
-EINVALand the vsyscall number on success so that you can get rid of returning it through the arg pointer?Then at the callsite you can do:
vsyscall_nr =
addr_to_vsyscall_nr(addr);
if (vsyscall_nr< 0)
warn_bad_vsyscall(...)Because I don't want a warning about ret being used without being initialized.
not if you preinit it...
Quoted 3 lines
With the code in this patch, the compiler is smart enough to figure out that either
vsyscall_nris 0, 1, or 2 or that the EINVAL branch is taken. I'll see if it works the other way.
here's what i mean, I changed your patch a bit:
--
Quoted 1 line
From
2bb6c706cd6cf49ce5baec670bd01b53d2ea6f19Mon Sep 17 00:00:00 2001
From: Andy Lutomirski <[email protected]>
Date: Sat, 9 Jul 2011 23:22:08 -0400
Subject: [PATCH] x86-64: Improve vsyscall emulation CS and RIP handling
Two fixes here:
- Send SIGSEGV if called from compat code or with a funny CS.
- Don't BUG on impossible addresses.
This patch also removes an unused variable.
--- arch/x86/include/asm/vsyscall.h | 12 ------- arch/x86/kernel/vsyscall_64.c | 62 +++++++++++++++++++++++++------------- 2 files changed, 41 insertions(+), 33 deletions(-)
arch/x86/include/asm/vsyscall.h +0 -12
@@ -31,18 +31,6 @@ extern struct timezone sys_tz;extern void map_vsyscall (void ); -/* Emulation */ - -static inline bool is_vsyscall_entry (unsigned long addr) -{ -return (addr & ~0xC00UL ) ==VSYSCALL_START ; -} - -static inline int vsyscall_entry_nr (unsigned long addr) -{ -return (addr &0xC00UL ) >>10 ; -} -#endif /* __KERNEL__ */ #endif /* _ASM_X86_VSYSCALL_H */
arch/x86/kernel/vsyscall_64.c +41 -21
111 lines not on this page · raw message
Signature · 8 lines
1.7.5.3.401.gfb674
--
Regards/Gruss,
Boris.Re: [PATCH v2 1/8] x86-64: Improve vsyscall emulation CS and RIP handling
Quoted 7 lines · Borislav Petkov
Quoted 6 lines · Andrew Lutomirski
Quoted 6 lines
I'm wondering: why don't you make this function return negative value on error, i.e.
-EINVALand the vsyscall number on success so that you can get rid of returning it through the arg pointer?Then at the callsite you can do:
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
vsyscall_nr =
addr_to_vsyscall_nr(addr);
if (vsyscall_nr< 0)
warn_bad_vsyscall(...)Because I don't want a warning about ret being used without being initialized.
not if you preinit it...
I kind of like that warning as a sanity check, and preiniting it grates against my irrational desire to over-optimize :)
Quoted 6 lines
Quoted 4 lines
With the code in this patch, the compiler is smart enough to figure out that either
vsyscall_nris 0, 1, or 2 or that the EINVAL branch is taken. I'll see if it works the other way.here's what i mean, I changed your patch a bit:
How about this:
static int addr_to_vsyscall_nr(unsigned long addr)
{
int nr;
if ((addr & ~0xC00UL) != VSYSCALL_START)
return -EINVAL;
nr = (addr & 0xC00UL) >> 10;
if (nr >= 3)
return -EINVAL;
return nr;
}...
int vsyscall_nr;
...
vsyscall_nr = addr_to_vsyscall_nr(regs->ip - 2);
if (vsyscall_nr < 0) {
warn_bad_vsyscall(KERN_WARNING, regs,
"illegal int 0xcc (exploit attempt?)");
goto sigsegv;
}gcc 4.6 at least does not warn.
Also, IRQ disabling was still mismatched in the sigsegv path. I'll fix that as well.
--Andy
Re: [PATCH v2 1/8] x86-64: Improve vsyscall emulation CS and RIP handling
Quoted 12 lines · Andrew Lutomirski
Quoted 11 lines · Borislav Petkov
Quoted 10 lines · Andrew Lutomirski
Quoted 9 lines
I'm wondering: why don't you make this function return negative value on error, i.e.
-EINVALand the vsyscall number on success so that you can get rid of returning it through the arg pointer?Then at the callsite you can do:
vsyscall_nr =
addr_to_vsyscall_nr(addr);
if (vsyscall_nr< 0)
warn_bad_vsyscall(...)
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Because I don't want a warning about ret being used without being initialized.
not if you preinit it...
I kind of like that warning as a sanity check, and preiniting it grates against my irrational desire to over-optimize :)
:-)
Quoted 29 lines
Quoted 6 lines
Quoted 4 lines
With the code in this patch, the compiler is smart enough to figure out that either
vsyscall_nris 0, 1, or 2 or that the EINVAL branch is taken. I'll see if it works the other way.here's what i mean, I changed your patch a bit:
How about this:
static int
addr_to_vsyscall_nr(unsigned long addr){ int nr; if ((addr & ~0xC00UL) != VSYSCALL_START) return -EINVAL; nr = (addr & 0xC00UL) >> 10; if (nr >= 3) return -EINVAL; return nr; }...
int vsyscall_nr;...
Quoted 8 lines
vsyscall_nr = addr_to_vsyscall_nr(regs->ip - 2); if (vsyscall_nr < 0) { warn_bad_vsyscall(KERN_WARNING, regs, "illegal int 0xcc (exploit attempt?)"); goto sigsegv; }gcc 4.6 at least does not warn.
Yep, looks good.
Quoted 2 lines
Also, IRQ disabling was still mismatched in the sigsegv path. I'll fix that as well.
oh yeah.
Thanks.
Signature · 8 lines
Regards/Gruss, Boris. Advanced Micro Devices GmbH Einsteinring 24, 85609 Dornach GM: Alberto Bozzo Reg: Dornach, Landkreis Muenchen HRB Nr. 43632 WEEE Registernr: 129 19551
Re: [PATCH v2 1/8] x86-64: Improve vsyscall emulation CS and RIP handling
In reply to Andrew LutomirskiQuoted 4 lines · Andrew Lutomirski
Also, IRQ disabling was still mismatched in the sigsegv path. I'll fix that as well.
Just to make sure we're clear, we're still waiting for a new version of this patch, right?
-hpa
Re: [PATCH v2 1/8] x86-64: Improve vsyscall emulation CS and RIP handling
Quoted 8 lines · H. Peter Anvin
Quoted 4 lines · Andrew Lutomirski
Also, IRQ disabling was still mismatched in the sigsegv path. I'll fix that as well.
Just to make sure we're clear, we're still waiting for a new version of this patch, right?
Yes. I'm testing it right now and then I'll email it out.
--Andy
Quoted 3 lines
-hpa
PATCH v2 2/8 x86: Make alternative instruction pointers relative
In reply to Andy LutomirskiThis save a few bytes on x86-64 and means that future patches can apply alternatives to unrelocated code.
--- arch/x86/include/asm/alternative-asm.h | 4 ++-- arch/x86/include/asm/alternative.h | 8 ++++---- arch/x86/include/asm/cpufeature.h | 8 ++++---- arch/x86/kernel/alternative.c | 21 +++++++++++++-------- arch/x86/lib/copy_page_64.S | 9 +++------ arch/x86/lib/memmove_64.S | 11 +++++------ 6 files changed, 31 insertions(+), 30 deletions(-)
arch/x86/include/asm/alternative-asm.h +2 -2
@@ -17,8 +17,8 @@ .macro altinstruction_entry orig alt feature orig_len alt_len .align 8 - .quad \orig - .quad \alt + .long \orig - . + .long \alt - . .word \feature .byte \orig_len .byte \alt_len
arch/x86/include/asm/alternative.h +4 -4
@@ -43,8 +43,8 @@#endif struct alt_instr { -u8 *instr;/* original instruction */ -u8 *replacement; +s32 instr_offset;/* original instruction */ +s32 repl_offset;/* offset to replacement instruction */ u16 cpuid;/* cpuid bit set for replacement */ u8 instrlen;/* length of original instruction */ u8 replacementlen;/* length of new instruction, <= instrlen */
@@ -84,8 +84,8 @@ static inline int alternatives_text_reserved(void *start, void *end)"661:\n\t" oldinstr"\n662:\n" \".section .altinstructions,\"a\"\n" \ _ASM_ALIGN"\n" \ - _ASM_PTR"661b\n" /* label */ \ - _ASM_PTR"663f\n" /* new instruction */ \ +" .long 661b - .\n" /* label */ \ +" .long 663f - .\n" /* new instruction */ \ " .word " __stringify(feature) "\n" /* feature bit */ \" .byte 662b-661b\n" /* sourcelen */ \" .byte 664f-663f\n" /* replacementlen */ \
arch/x86/include/asm/cpufeature.h +4 -4
@@ -331,8 +331,8 @@ static __always_inline __pure bool __static_cpu_has(u16 bit)"2:\n" ".section .altinstructions,\"a\"\n" _ASM_ALIGN"\n" - _ASM_PTR"1b\n" - _ASM_PTR"0\n" /* no replacement */ +" .long 1b - .\n" +" .long 0\n" /* no replacement */ " .word %P0\n" /* feature bit */ " .byte 2b - 1b\n" /* source len */ " .byte 0\n" /* replacement len */
@@ -349,8 +349,8 @@ static __always_inline __pure bool __static_cpu_has(u16 bit)"2:\n" ".section .altinstructions,\"a\"\n" _ASM_ALIGN"\n" - _ASM_PTR"1b\n" - _ASM_PTR"3f\n" +" .long 1b - .\n" +" .long 3f - .\n" " .word %P1\n" /* feature bit */ " .byte 2b - 1b\n" /* source len */ " .byte 4f - 3f\n" /* replacement len */
arch/x86/kernel/alternative.c +13 -8
@@ -263,6 +263,7 @@ void __init_or_module apply_alternatives(struct alt_instr *start, struct alt_instr *end) {struct alt_instr *a; +u8 *instr, *replacement;u8 insnbuf[MAX_PATCH_LEN ];DPRINTK ("%s: alt table %p -> %p\n" , __func__, start, end);
@@ -276,25 +277,29 @@ void __init_or_module apply_alternatives(struct alt_instr *start,* order. */ for (a = start; a < end; a++) { -u8 *instr = a->instr; + instr = (u8 *)&a->instr_offset + a->instr_offset; + replacement = (u8 *)&a->repl_offset + a->repl_offset;BUG_ON (a->replacementlen > a->instrlen);BUG_ON (a->instrlen >sizeof (insnbuf));BUG_ON (a->cpuid >=NCAPINTS *32 );if (!boot_cpu_has (a->cpuid))continue ; + +memcpy (insnbuf, replacement, a->replacementlen); + +/* 0xe8 is a relative jump; fix the offset. */ +if (*insnbuf ==0xe8 && a->replacementlen ==5 ) + *(s32 *)(insnbuf +1 ) += replacement - instr; + +add_nops (insnbuf + a->replacementlen, + a->instrlen - a->replacementlen); +#ifdef CONFIG_X86_64 /* vsyscall code is not mapped yet. resolve it manually. */ if (instr >= (u8 *)VSYSCALL_START && instr < (u8 *)VSYSCALL_END ) { instr =__va (instr - (u8 *)VSYSCALL_START + (u8 *)__pa_symbol (&__vsyscall_0)); -DPRINTK ("%s: vsyscall fixup: %p => %p\n" , - __func__, a->instr, instr); }#endif -memcpy (insnbuf, a->replacement, a->replacementlen); -if (*insnbuf ==0xe8 && a->replacementlen ==5 ) - *(s32 *)(insnbuf +1 ) += a->replacement - a->instr; -add_nops (insnbuf + a->replacementlen, - a->instrlen - a->replacementlen);text_poke_early (instr, insnbuf, a->instrlen); } }
arch/x86/lib/copy_page_64.S +3 -6
@@ -2,6 +2,7 @@ #include <linux/linkage.h> #include <asm/dwarf2.h> +#include <asm/alternative-asm.h> ALIGN copy_page_c:
@@ -110,10 +111,6 @@ ENDPROC(copy_page) 2: .previous .section .altinstructions,"a" - .align 8 - .quad copy_page - .quad 1b - .word X86_FEATURE_REP_GOOD - .byte .Lcopy_page_end - copy_page - .byte 2b - 1b + altinstruction_entry copy_page, 1b, X86_FEATURE_REP_GOOD, \ + .Lcopy_page_end-copy_page, 2b-1b .previous
arch/x86/lib/memmove_64.S +5 -6
@@ -9,6 +9,7 @@ #include <linux/linkage.h> #include <asm/dwarf2.h> #include <asm/cpufeature.h> +#include <asm/alternative-asm.h> #undef memmove
@@ -214,11 +215,9 @@ ENTRY(memmove) .previous .section .altinstructions,"a" - .align 8 - .quad .Lmemmove_begin_forward - .quad .Lmemmove_begin_forward_efs - .word X86_FEATURE_ERMS - .byte .Lmemmove_end_forward-.Lmemmove_begin_forward - .byte .Lmemmove_end_forward_efs-.Lmemmove_begin_forward_efs + altinstruction_entry .Lmemmove_begin_forward, \ + .Lmemmove_begin_forward_efs,X86_FEATURE_ERMS, \ + .Lmemmove_end_forward-.Lmemmove_begin_forward, \ + .Lmemmove_end_forward_efs-.Lmemmove_begin_forward_efs .previous ENDPROC(memmove)
Signature · 1 line
1.7.6
PATCH v2 3/8 x86-64: Allow alternative patching in the vDSO
In reply to Andy LutomirskiThis code is short enough and different enough from the module loader that it's not worth trying to share anything.
--- arch/x86/vdso/vma.c | 30 ++++++++++++++++++++++++++++++ 1 files changed, 30 insertions(+), 0 deletions(-)
arch/x86/vdso/vma.c +30 -0
@@ -23,11 +23,41 @@ extern unsigned short vdso_sync_cpuid;static struct page **vdso_pages;static unsigned vdso_size; +static void patch_vdso (void *vdso,size_t len) +{ +Elf64_Ehdr *hdr = vdso; +Elf64_Shdr *sechdrs, *alt_sec =0 ; +char *secstrings; +void *alt_data; +int i; + +BUG_ON (len <sizeof (Elf64_Ehdr)); +BUG_ON (memcmp (hdr->e_ident,ELFMAG ,SELFMAG ) !=0 ); + + sechdrs = (void *)hdr + hdr->e_shoff; + secstrings = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset; + +for (i =1 ; i < hdr->e_shnum; i++) { +Elf64_Shdr *shdr = &sechdrs[i]; +if (!strcmp (secstrings + shdr->sh_name,".altinstructions" )) { + alt_sec = shdr; + goto found; + } + } +return ;/* nothing to patch */ + +found: + alt_data = (void *)hdr + alt_sec->sh_offset; +apply_alternatives (alt_data, alt_data + alt_sec->sh_size); +} +static int __init init_vdso_vars (void ) {int npages = (vdso_end - vdso_start +PAGE_SIZE -1 ) /PAGE_SIZE ;int i; +patch_vdso (vdso_start, vdso_end - vdso_start); + vdso_size = npages <<PAGE_SHIFT ; vdso_pages =kmalloc (sizeof (struct page *) * npages,GFP_KERNEL );if (!vdso_pages)
Signature · 1 line
1.7.6
Re: [PATCH v2 3/8] x86-64: Allow alternative patching in the vDSO
Quoted 13 lines · Andy Lutomirski
This code is short enough and different enough from the module loader that it's not worth trying to share anything.
Signed-off-by Andy Lutomirski <[email protected]>---
arch/x86/vdso/vma.c | 30 ++++++++++++++++++++++++++++++1 files changed, 30 insertions(+), 0 deletions(-)arch/x86/vdso/vma.c +0
-0
arch/x86/vdso/vma.c extern unsigned short vdso_sync_cpuid; · line 26
Quoted 4 lines
arch/x86/vdso/vma.c +1
-0@@ -23,11 +23,41 @@ extern unsigned short vdso_sync_cpuid; static struct page **vdso_pages; static unsigned vdso_size; +static void patch_vdso (void *vdso,size_t len)
I think, patch_vdso should mark with __init. Since it's been only called from an __init function. We might hit a section mismatch, even if we don't hit section mismatch, the will function get removed after system boot - saves some .text size.
Quoted 11 lines
arch/x86/vdso/vma.c +11
-0+{ + Elf64_Ehdr *hdr = vdso; + Elf64_Shdr *sechdrs, *alt_sec = 0; + char *secstrings; + void *alt_data; + int i; + + BUG_ON(len < sizeof(Elf64_Ehdr)); + BUG_ON(memcmp(hdr->e_ident, ELFMAG, SELFMAG) != 0); + + sechdrs = (void *)hdr + hdr->e_shoff;
arch/x86/vdso/vma.c extern unsigned short vdso_sync_cpuid; · line 47
Quoted 10 lines
arch/x86/vdso/vma.c +10
-0+ secstrings = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset; + + for (i = 1; i < hdr->e_shnum; i++) { + Elf64_Shdr *shdr = &sechdrs[i]; + if (!strcmp(secstrings + shdr->sh_name, ".altinstructions")) { + alt_sec = shdr; + goto found; + } + } + return; /* nothing to patch */
If there's nothing to patch, I think (perhaps) - it would be nice to let user know through printk.
Thanks,
Rakib.
Re: [PATCH v2 3/8] x86-64: Allow alternative patching in the vDSO
Quoted 8 lines · Rakib Mullick
Quoted 1 line · Andy Lutomirski
+static void
patch_vdso(void *vdso, size_t len)I think,
patch_vdsoshould mark with__init. Since it's been only called from an__initfunction. We might hit a section mismatch, even if we don't hit section mismatch, the will function get removed after system boot - saves some .text size.
Sure. It won't make any difference, because my compiler at least is smart enough to inline the whole function into init_vdso_vars, but it's the right thing to do anyway.
Quoted 4 lines
Quoted 1 line
+ return; /* nothing to patch */
If there's nothing to patch, I think (perhaps) - it would be nice to let user know through printk.
Will do. I verified it myself by dumping the vdso image from memory and confirming that it got patched correctly, but if the code ever bitrots it'll be nice to have the warning.
--Andy
PATCH v2 4/8 x86-64: Add --no-undefined to vDSO build
In reply to Andy LutomirskiThis gives much nicer diagnostics when something goes wrong. It's supported at least as far back as binutils 2.15.
--- arch/x86/vdso/Makefile | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
arch/x86/vdso/Makefile +1 -0
@@ -26,6 +26,7 @@ targets += vdso.so vdso.so.dbg vdso.lds $(vobjs-y) export CPPFLAGS_vdso.lds += -P -C VDSO_LDFLAGS_vdso.lds = -m64 -Wl,-soname=linux-vdso.so.1 \ + -Wl,--no-undefined \ -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096 $(obj)/vdso.o: $(src)/vdso.S $(obj)/vdso.so
Signature · 1 line
1.7.6
PATCH v2 5/8 clocksource: Replace vread with generic arch data
In reply to Andy LutomirskiThe vread field was bloating struct clocksource everywhere except x86_64, and I want to change the way this works on x86_64, so let's split it out into per-arch data.
--- arch/x86/include/asm/clocksource.h | 16 ++++++++++++++++ arch/x86/kernel/hpet.c | 2 +- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vsyscall_64.c | 2 +- include/asm-generic/clocksource.h | 4 ++++ include/linux/clocksource.h | 10 ++++++++-- 6 files changed, 31 insertions(+), 5 deletions(-) create mode 100644 arch/x86/include/asm/clocksource.h create mode 100644 include/asm-generic/clocksource.h
arch/x86/include/asm/clocksource.h new file +16 -0
@@ -0,0 +1,16 @@ +/* x86-specific clocksource additions */ + +#ifndef _ASM_X86_CLOCKSOURCE_H +#define _ASM_X86_CLOCKSOURCE_H + +#ifdef CONFIG_X86_64 + +#define __ARCH_HAS_CLOCKSOURCE_DATA + +struct arch_clocksource_data { +cycle_t (*vread)(void ); +}; + +#endif /* CONFIG_X86_64 */ + +#endif /* _ASM_X86_CLOCKSOURCE_H */
arch/x86/kernel/hpet.c +1 -1
@@ -753,7 +753,7 @@ static struct clocksource clocksource_hpet = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .resume = hpet_resume_counter, #ifdef CONFIG_X86_64 - .vread = vread_hpet, + .archdata = { .vread = vread_hpet },#endif };
arch/x86/kernel/tsc.c +1 -1
@@ -777,7 +777,7 @@ static struct clocksource clocksource_tsc = { .flags = CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_MUST_VERIFY, #ifdef CONFIG_X86_64 - .vread = vread_tsc, + .archdata = { .vread = vread_tsc },#endif };
arch/x86/kernel/vsyscall_64.c +1 -1
@@ -74,7 +74,7 @@ void update_vsyscall(struct timespec *wall_time, struct timespec *wtm,write_seqlock_irqsave (&vsyscall_gtod_data.lock, flags);/* copy vsyscall data */ - vsyscall_gtod_data.clock.vread = clock->vread; + vsyscall_gtod_data.clock.vread = clock->archdata.vread; vsyscall_gtod_data.clock.cycle_last = clock->cycle_last; vsyscall_gtod_data.clock.mask = clock->mask; vsyscall_gtod_data.clock.mult = mult;
include/asm-generic/clocksource.h new file +4 -0
@@ -0,0 +1,4 @@ +/* +* Architectures should override this file to add private userspace +* clock magic if needed. +*/
include/linux/clocksource.h +8 -2
@@ -22,6 +22,8 @@typedef u64 cycle_t ;struct clocksource ; +#include <asm/clocksource.h> +/** * struct cyclecounter - hardware abstraction for a free running counter * Provides completely state-free accessors to the underlying hardware.
@@ -153,7 +155,7 @@ extern u64 timecounter_cyc2time(struct timecounter *tc,* @shift: cycle to nanosecond divisor (power of two) * @max_idle_ns: max idle time permitted by the clocksource (nsecs) * @flags: flags describing special properties -* @vread: vsyscall based read +* @archdata: arch-specific data * @suspend: suspend function for the clocksource, if necessary * @resume: resume function for the clocksource, if necessary */
@@ -175,10 +177,14 @@ struct clocksource {#else #define CLKSRC_FSYS_MMIO_SET (mmio, addr) do { } while (0)#endif + +#ifdef __ARCH_HAS_CLOCKSOURCE_DATA +struct arch_clocksource_data archdata; +#endif +const char *name;struct list_head list;int rating; -cycle_t (*vread)(void);int (*enable)(struct clocksource *cs);void (*disable)(struct clocksource *cs);unsigned long flags;
Signature · 1 line
1.7.6
PATCH v2 6/8 x86-64: Move vread_tsc and vread_hpet into the vDSO
In reply to Andy LutomirskiThe vsyscall page now consists entirely of trap instructions.
--- arch/x86/include/asm/clocksource.h | 6 +++- arch/x86/include/asm/tsc.h | 4 --- arch/x86/include/asm/vgtod.h | 2 +- arch/x86/include/asm/vsyscall.h | 4 --- arch/x86/kernel/Makefile | 7 +---- arch/x86/kernel/alternative.c | 8 ----- arch/x86/kernel/hpet.c | 9 +----- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 3 -- arch/x86/kernel/vread_tsc_64.c | 36 ------------------------- arch/x86/kernel/vsyscall_64.c | 2 +- arch/x86/vdso/vclock_gettime.c | 52 +++++++++++++++++++++++++++++++---- 12 files changed, 56 insertions(+), 79 deletions(-) delete mode 100644 arch/x86/kernel/vread_tsc_64.c
arch/x86/include/asm/clocksource.h +5 -1
@@ -7,8 +7,12 @@#define __ARCH_HAS_CLOCKSOURCE_DATA +#define VCLOCK_NONE 0/* No vDSO clock available. */ +#define VCLOCK_TSC 1/* vDSO should use vread_tsc. */ +#define VCLOCK_HPET 2/* vDSO should use vread_hpet. */ +struct arch_clocksource_data { -cycle_t (*vread)(void ); +int vclock_mode; };#endif /* CONFIG_X86_64 */
arch/x86/include/asm/tsc.h +0 -4
@@ -51,10 +51,6 @@ extern int unsynchronized_tsc(void);extern int check_tsc_unstable (void );extern unsigned long native_calibrate_tsc (void ); -#ifdef CONFIG_X86_64 -extern cycles_t vread_tsc (void ); -#endif -/* * Boot-time check whether the TSCs are synchronized across * all CPUs/cores:
arch/x86/include/asm/vgtod.h +1 -1
@@ -13,7 +13,7 @@ struct vsyscall_gtod_data {struct timezone sys_tz;struct {/* extract of a clocksource struct */ -cycle_t (*vread)(void ); +int vclock_mode;cycle_t cycle_last;cycle_t mask;u32 mult;
arch/x86/include/asm/vsyscall.h +0 -4
@@ -16,10 +16,6 @@ enum vsyscall_num {#ifdef __KERNEL__#include <linux/seqlock.h> -/* Definitions for CONFIG_GENERIC_TIME definitions */ -#define __vsyscall_fn \ - __attribute__ ((unused, __section__(".vsyscall_fn"))) notrace -#define VGETCPU_RDTSCP 1#define VGETCPU_LSL 2
arch/x86/kernel/Makefile +1 -6
@@ -24,17 +24,12 @@ endif nostackp := $(call cc-option, -fno-stack-protector) CFLAGS_vsyscall_64.o := $(PROFILING) -g0 $(nostackp) CFLAGS_hpet.o := $(nostackp) -CFLAGS_vread_tsc_64.o := $(nostackp) CFLAGS_paravirt.o := $(nostackp) GCOV_PROFILE_vsyscall_64.o := n GCOV_PROFILE_hpet.o := n GCOV_PROFILE_tsc.o := n -GCOV_PROFILE_vread_tsc_64.o := n GCOV_PROFILE_paravirt.o := n -# vread_tsc_64 is hot and should be fully optimized: -CFLAGS_REMOVE_vread_tsc_64.o = -pg -fno-optimize-sibling-calls - obj-y := process_$(BITS).o signal.o entry_$(BITS).o obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o ldt.o dumpstack.o
@@ -43,7 +38,7 @@ obj-$(CONFIG_IRQ_WORK) += irq_work.o obj-y += probe_roms.o obj-$(CONFIG_X86_32) += sys_i386_32.o i386_ksyms_32.o obj-$(CONFIG_X86_64) += sys_x86_64.o x8664_ksyms_64.o -obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o vread_tsc_64.o +obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o obj-$(CONFIG_X86_64) += vsyscall_emu_64.o obj-y += bootflag.o e820.o obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
arch/x86/kernel/alternative.c +0 -8
@@ -14,7 +14,6 @@#include <asm/pgtable.h> #include <asm/mce.h> #include <asm/nmi.h> -#include <asm/vsyscall.h> #include <asm/cacheflush.h> #include <asm/tlbflush.h> #include <asm/io.h>
@@ -250,7 +249,6 @@ static void __init_or_module add_nops(void *insns, unsigned int len)extern struct alt_instr __alt_instructions[], __alt_instructions_end[];extern s32 __smp_locks[], __smp_locks_end[]; -extern char __vsyscall_0;void *text_poke_early (void *addr,const void *opcode,size_t len);/* Replace instructions with better alternatives for this CPU type.
@@ -294,12 +292,6 @@ void __init_or_module apply_alternatives(struct alt_instr *start,add_nops (insnbuf + a->replacementlen, a->instrlen - a->replacementlen); -#ifdef CONFIG_X86_64 -/* vsyscall code is not mapped yet. resolve it manually. */ -if (instr >= (u8 *)VSYSCALL_START && instr < (u8 *)VSYSCALL_END ) { - instr =__va (instr - (u8 *)VSYSCALL_START + (u8 *)__pa_symbol (&__vsyscall_0)); - } -#endif text_poke_early (instr, insnbuf, a->instrlen); } }
arch/x86/kernel/hpet.c +1 -8
@@ -738,13 +738,6 @@ static cycle_t read_hpet(struct clocksource *cs)return (cycle_t )hpet_readl (HPET_COUNTER ); } -#ifdef CONFIG_X86_64 -static cycle_t __vsyscall_fn vread_hpet(void) -{ -return readl ((const void __iomem *)fix_to_virt (VSYSCALL_HPET ) +0xf0 ); -} -#endif -static struct clocksource clocksource_hpet = { .name ="hpet" , .rating =250 ,
@@ -753,7 +746,7 @@ static struct clocksource clocksource_hpet = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .resume = hpet_resume_counter, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_hpet }, + .archdata = { .vclock_mode = VCLOCK_HPET },#endif };
arch/x86/kernel/tsc.c +1 -1
@@ -777,7 +777,7 @@ static struct clocksource clocksource_tsc = { .flags = CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_MUST_VERIFY, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_tsc }, + .archdata = { .vclock_mode = VCLOCK_TSC },#endif };
arch/x86/kernel/vmlinux.lds.S +0 -3
@@ -169,9 +169,6 @@ SECTIONS .vsyscall : AT(VLOAD(.vsyscall)) { *(.vsyscall_0) - . = ALIGN(L1_CACHE_BYTES); - *(.vsyscall_fn) - . = 1024; *(.vsyscall_1)
arch/x86/kernel/vread_tsc_64.c deleted +0 -36
@@ -1,36 +0,0 @@ -/* This code runs in userspace. */ - -#define DISABLE_BRANCH_PROFILING -#include <asm/vgtod.h> - -notrace cycle_t __vsyscall_fn vread_tsc(void) -{ -cycle_t ret; -u64 last; - -/* -* Empirically, a fence (of type that depends on the CPU) -* before rdtsc is enough to ensure that rdtsc is ordered -* with respect to loads. The various CPU manuals are unclear -* as to whether rdtsc can be reordered with later loads, -* but no one has ever seen it happen. -*/ -rdtsc_barrier (); - ret = (cycle_t )vget_cycles (); - - last =VVAR (vsyscall_gtod_data).clock.cycle_last; - -if (likely (ret >= last)) -return ret; - -/* -* GCC likes to generate cmov here, but this branch is extremely -* predictable (it's just a funciton of time and the likely is -* very likely) and there's a data dependence, so force GCC -* to generate a branch instead. I don't barrier() because -* we don't actually need a barrier, and if this function -* ever gets inlined it will generate worse code. -*/ - asmvolatile ("" ); -return last; -}
arch/x86/kernel/vsyscall_64.c +1 -1
@@ -74,7 +74,7 @@ void update_vsyscall(struct timespec *wall_time, struct timespec *wtm,write_seqlock_irqsave (&vsyscall_gtod_data.lock, flags);/* copy vsyscall data */ - vsyscall_gtod_data.clock.vread = clock->archdata.vread; + vsyscall_gtod_data.clock.vclock_mode = clock->archdata.vclock_mode; vsyscall_gtod_data.clock.cycle_last = clock->cycle_last; vsyscall_gtod_data.clock.mask = clock->mask; vsyscall_gtod_data.clock.mult = mult;
arch/x86/vdso/vclock_gettime.c +46 -6
79 lines not on this page · raw message
Signature · 1 line
1.7.6
PATCH v2 7/8 ia64: Replace clocksource.fsys_mmio with generic arch data
In reply to Andy LutomirskiNow that clocksource.archdata is available, use it for ia64-specific code.
--- arch/ia64/include/asm/clocksource.h | 12 ++++++++++++ arch/ia64/kernel/cyclone.c | 2 +- arch/ia64/kernel/time.c | 2 +- arch/ia64/sn/kernel/sn2/timer.c | 2 +- drivers/char/hpet.c | 2 +- include/linux/clocksource.h | 7 ------- 6 files changed, 16 insertions(+), 11 deletions(-) create mode 100644 arch/ia64/include/asm/clocksource.h
arch/ia64/include/asm/clocksource.h new file +12 -0
@@ -0,0 +1,12 @@ +/* IA64-specific clocksource additions */ + +#ifndef _ASM_IA64_CLOCKSOURCE_H +#define _ASM_IA64_CLOCKSOURCE_H + +#define __ARCH_HAS_CLOCKSOURCE_DATA + +struct arch_clocksource_data { +void *fsys_mmio;/* used by fsyscall asm code */ +}; + +#endif /* _ASM_IA64_CLOCKSOURCE_H */
arch/ia64/kernel/cyclone.c +1 -1
@@ -115,7 +115,7 @@ int __init init_cyclone_clock(void) }/* initialize last tick */ cyclone_mc = cyclone_timer; - clocksource_cyclone.fsys_mmio = cyclone_timer; + clocksource_cyclone.archdata.fsys_mmio = cyclone_timer;clocksource_register_hz (&clocksource_cyclone,CYCLONE_TIMER_FREQ );return 0 ;
arch/ia64/kernel/time.c +1 -1
@@ -468,7 +468,7 @@ void update_vsyscall(struct timespec *wall, struct timespec *wtm, fsyscall_gtod_data.clk_mask = c->mask; fsyscall_gtod_data.clk_mult = mult; fsyscall_gtod_data.clk_shift = c->shift; - fsyscall_gtod_data.clk_fsys_mmio = c->fsys_mmio; + fsyscall_gtod_data.clk_fsys_mmio = c->archdata.fsys_mmio; fsyscall_gtod_data.clk_cycle_last = c->cycle_last;/* copy kernel time structures */
arch/ia64/sn/kernel/sn2/timer.c +1 -1
@@ -54,7 +54,7 @@ ia64_sn_udelay (unsigned long usecs)void __init sn_timer_init (void ) { - clocksource_sn2.fsys_mmio =RTC_COUNTER_ADDR ; + clocksource_sn2.archdata.fsys_mmio =RTC_COUNTER_ADDR ;clocksource_register_hz (&clocksource_sn2, sn_rtc_cycles_per_second); ia64_udelay = &ia64_sn_udelay;
drivers/char/hpet.c +1 -1
@@ -952,7 +952,7 @@ int hpet_alloc(struct hpet_data *hdp)#ifdef CONFIG_IA64 if (!hpet_clocksource) { hpet_mctr = (void __iomem *)&hpetp->hp_hpet->hpet_mc; -CLKSRC_FSYS_MMIO_SET (clocksource_hpet.fsys_mmio, hpet_mctr); + clocksource_hpet.archdata.fsys_mmio = hpet_mctr;clocksource_register_hz (&clocksource_hpet, hpetp->hp_tick_freq); hpetp->hp_clocksource = &clocksource_hpet; hpet_clocksource = &clocksource_hpet;
include/linux/clocksource.h +0 -7
@@ -171,13 +171,6 @@ struct clocksource {u32 shift;u64 max_idle_ns; -#ifdef CONFIG_IA64 -void *fsys_mmio;/* used by fsyscall asm code */ -#define CLKSRC_FSYS_MMIO_SET (mmio, addr) ((mmio) = (addr)) -#else -#define CLKSRC_FSYS_MMIO_SET (mmio, addr) do { } while (0) -#endif -#ifdef __ARCH_HAS_CLOCKSOURCE_DATAstruct arch_clocksource_data archdata;#endif
Signature · 1 line
1.7.6
PATCH v2 8/8 Document the vDSO and add a reference parser
In reply to Andy LutomirskiIt turns out that parsing the vDSO is nontrivial if you don't already have an ELF dynamic loader around. So document it in Documentation/ABI and add a reference CC0-licenced parser.
This code is dedicated to Go issue 1933:
http://code.google.com/p/go/issues/detail?id=1933
--- Documentation/ABI/stable/vdso | 27 ++++ Documentation/vDSO/parse_vdso.c | 256 +++++++++++++++++++++++++++++++++++++++ Documentation/vDSO/vdso_test.c | 112 +++++++++++++++++ 3 files changed, 395 insertions(+), 0 deletions(-) create mode 100644 Documentation/ABI/stable/vdso create mode 100644 Documentation/vDSO/parse_vdso.c create mode 100644 Documentation/vDSO/vdso_test.c
Documentation/ABI/stable/vdso new file +27 -0
@@ -0,0 +1,27 @@ +On some architectures, when the kernel loads any userspace program it +maps an ELF DSO into that program's address space. This DSO is called +the vDSO and it often contains useful and highly-optimized alternatives +to real syscalls. + +These functions are called just like ordinary C function according to +your platform's ABI. Call them from a sensible context. (For example, +if you set CS on x86 to something strange, the vDSO functions are +within their rights to crash.) In addition, if you pass a bad +pointer to a vDSO function, you might get SIGSEGV instead of -EFAULT. + +To find the DSO, parse the auxiliary vector passed to the program's +entry point. The AT_SYSINFO_EHDR entry will point to the vDSO. + +The vDSO uses symbol versioning; whenever you request a symbol from the +vDSO, specify the version you are expecting. + +Programs that dynamically link to glibc will use the vDSO automatically. +Otherwise, you can use the reference parser in Documentation/vDSO/parse_vdso.c. + +Unless otherwise noted, the set of symbols with any given version and the +ABI of those symbols is considered stable. It may vary across architectures, +though. + +(As of this writing, this ABI documentation as been confirmed for x86_64. + The maintainers of the other vDSO-using architectures should confirm + that it is correct for their architecture.)
Documentation/vDSO/parse_vdso.c new file +256 -0
256 lines not on this page · raw message
Documentation/vDSO/vdso_test.c new file +112 -0
112 lines not on this page · raw message
Signature · 1 line
1.7.6