x86-64 vDSO changes for 3.1
Subsystems (7)
v3 cover x86-64 vDSO changes for 3.1
This series applies to the x86/vdso branch of the -tip tree.
The first patch cleans up the vsyscall emulation code.
After the vsyscall emulation patches, the only real executable code left in the vsyscall page is vread_tsc and vread_hpet. That code is only called from the vDSO, so patches 2-6 move it into the vDSO.
vread_tsc() uses rdtsc_barrier(), which contains two alternatives. Patches 2 and 3 make alternative patching work in the vDSO. (This has a slightly odd side effect that the vDSO image dumped from memory doesn't quite match the debug version anymore, but it's hard to imagine that causing problems.)
Patch 4 fixes an annoyance I found while writing this code. If you introduce an undefined symbol into the vDSO, you get an unhelpful error message. ld is smart enough to give a nice error if you ask it to.
Patch 5 cleans up the architecture-specific part of struct clocksource. IA64 had its own ifdefed code in there, and the supposedly generic vread pointer was only used by x86-64. With the patch, each arch gets to set up its own private part of struct clocksource.
Patch 6 is the meat. It moves vread_tsc and vread_hpet into the vDSO where they belong, and it's a net deletion of code because it removes a bunch of magic needed to make regular functions accessible through the vsyscall page.
With patches 1-6 applied, every single byte in the vsyscall page is some sort of trap instruction.
Patches 7 and 8 are optional. Patch 7 changes IA64 to use the new arch gtod data. It presumably should not go in through the x86 tree. Patch 8 adds some vDSO documentation and a reference vDSO parser for user code to use. It's meant for projects that don't dynamically link to glibc (e.g. Go) but still want to call the vDSO. Someone who knows more about ELF than I should take a look.
*** Note to IA64 people: I have not even compile-tested this on IA64. ***
Changes from v2:
- Make vsyscall_nr decoding prettier and add a missing local_irq_disable in patch 1.
- Mark patch_vdso __init.
- Print a warning if patch_vdso does not find .altinstructions.
Changes from v1:
- Tidy up vDSO alternative patching (thanks, Borislav).
- Fix really dumb bugs in the IA64 stuff.
- Add the cleanup patch and the reference vDSO parser.
- Split the main IA-64 patch out.
Andy Lutomirski (8):
x86-64: Improve vsyscall emulation CS and RIP handling
x86: Make alternative instruction pointers relative
x86-64: Allow alternative patching in the vDSO
x86-64: Add --no-undefined to vDSO build
clocksource: Replace vread with generic arch data
x86-64: Move vread_tsc and vread_hpet into the vDSO
ia64: Replace clocksource.fsys_mmio with generic arch data Document the vDSO and add a reference parser
Documentation/ABI/stable/vdso | 27 ++++ Documentation/vDSO/parse_vdso.c | 256 ++++++++++++++++++++++++++++++++ Documentation/vDSO/vdso_test.c | 112 ++++++++++++++ arch/ia64/include/asm/clocksource.h | 12 ++ arch/ia64/kernel/cyclone.c | 2 +- arch/ia64/kernel/time.c | 2 +- arch/ia64/sn/kernel/sn2/timer.c | 2 +- arch/x86/include/asm/alternative-asm.h | 4 +- arch/x86/include/asm/alternative.h | 8 +- arch/x86/include/asm/clocksource.h | 20 +++ arch/x86/include/asm/cpufeature.h | 8 +- arch/x86/include/asm/tsc.h | 4 - arch/x86/include/asm/vgtod.h | 2 +- arch/x86/include/asm/vsyscall.h | 16 -- arch/x86/kernel/Makefile | 7 +- arch/x86/kernel/alternative.c | 23 ++-- arch/x86/kernel/hpet.c | 9 +- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 3 - arch/x86/kernel/vread_tsc_64.c | 36 ----- arch/x86/kernel/vsyscall_64.c | 63 +++++--- arch/x86/lib/copy_page_64.S | 9 +- arch/x86/lib/memmove_64.S | 11 +- arch/x86/vdso/Makefile | 1 + arch/x86/vdso/vclock_gettime.c | 52 ++++++- arch/x86/vdso/vma.c | 33 ++++ drivers/char/hpet.c | 2 +- include/asm-generic/clocksource.h | 4 + include/linux/clocksource.h | 13 +- 29 files changed, 595 insertions(+), 148 deletions(-) create mode 100644 Documentation/ABI/stable/vdso create mode 100644 Documentation/vDSO/parse_vdso.c create mode 100644 Documentation/vDSO/vdso_test.c create mode 100644 arch/ia64/include/asm/clocksource.h create mode 100644 arch/x86/include/asm/clocksource.h delete mode 100644 arch/x86/kernel/vread_tsc_64.c create mode 100644 include/asm-generic/clocksource.h
Signature · 1 line
1.7.6
PATCH v3 1/8 x86-64: Improve vsyscall emulation CS and RIP handling
Three fixes here:
- Send SIGSEGV if called from compat code or with a funny CS.
- Don't BUG on impossible addresses.
- Add a missing local_irq_disable.
This patch also removes an unused variable.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/include/asm/vsyscall.h | 12 ------- arch/x86/kernel/vsyscall_64.c | 61 ++++++++++++++++++++++++++------------- 2 files changed, 41 insertions(+), 32 deletions(-)
arch/x86/include/asm/vsyscall.h +0 -12
@@ -31,18 +31,6 @@ extern struct timezone sys_tz;extern void map_vsyscall (void ); -/* Emulation */ - -static inline bool is_vsyscall_entry (unsigned long addr) -{ -return (addr & ~0xC00UL ) ==VSYSCALL_START ; -} - -static inline int vsyscall_entry_nr (unsigned long addr) -{ -return (addr &0xC00UL ) >>10 ; -} -#endif /* __KERNEL__ */ #endif /* _ASM_X86_VSYSCALL_H */
arch/x86/kernel/vsyscall_64.c +41 -20
117 lines not on this page · raw message
Signature · 1 line
1.7.6
[tip:x86/vdso] x86-64: Improve vsyscall emulation CS and RIP handling
Commit-ID: c9712944b2a12373cb6ff8059afcfb7e826a6c54 Gitweb: http://git.kernel.org/tip/c9712944b2a12373cb6ff8059afcfb7e826a6c54 Author: Andy Lutomirski <[email protected]> AuthorDate: Wed, 13 Jul 2011 09:24:09 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 13 Jul 2011 11:22:55 -0700
x86-64: Improve vsyscall emulation CS and RIP handling
Three fixes here:
- Send SIGSEGV if called from compat code or with a funny CS.
- Don't BUG on impossible addresses.
- Add a missing local_irq_disable.
This patch also removes an unused variable.
Signed-off-by Andy Lutomirski <[email protected]>
Link http://lkml.kernel.org/r/6fb2b13ab39b743d1e4f466eef13425854912f7f.1310563276.git.luto@mit.edu open here
Signed-off-by H. Peter Anvin <[email protected]>
--- arch/x86/include/asm/vsyscall.h | 12 ------- arch/x86/kernel/vsyscall_64.c | 61 ++++++++++++++++++++++++++------------- 2 files changed, 41 insertions(+), 32 deletions(-)
arch/x86/include/asm/vsyscall.h +0 -12
@@ -31,18 +31,6 @@ extern struct timezone sys_tz;extern void map_vsyscall (void ); -/* Emulation */ - -static inline bool is_vsyscall_entry (unsigned long addr) -{ -return (addr & ~0xC00UL ) ==VSYSCALL_START ; -} - -static inline int vsyscall_entry_nr (unsigned long addr) -{ -return (addr &0xC00UL ) >>10 ; -} -#endif /* __KERNEL__ */ #endif /* _ASM_X86_VSYSCALL_H */
arch/x86/kernel/vsyscall_64.c +41 -20
117 lines not on this page · raw message
PATCH v3 2/8 x86: Make alternative instruction pointers relative
In reply to Andy LutomirskiThis save a few bytes on x86-64 and means that future patches can apply alternatives to unrelocated code.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/include/asm/alternative-asm.h | 4 ++-- arch/x86/include/asm/alternative.h | 8 ++++---- arch/x86/include/asm/cpufeature.h | 8 ++++---- arch/x86/kernel/alternative.c | 21 +++++++++++++-------- arch/x86/lib/copy_page_64.S | 9 +++------ arch/x86/lib/memmove_64.S | 11 +++++------ 6 files changed, 31 insertions(+), 30 deletions(-)
arch/x86/include/asm/alternative-asm.h +2 -2
@@ -17,8 +17,8 @@ .macro altinstruction_entry orig alt feature orig_len alt_len .align 8 - .quad \orig - .quad \alt + .long \orig - . + .long \alt - . .word \feature .byte \orig_len .byte \alt_len
arch/x86/include/asm/alternative.h +4 -4
@@ -43,8 +43,8 @@#endif struct alt_instr { -u8 *instr;/* original instruction */ -u8 *replacement; +s32 instr_offset;/* original instruction */ +s32 repl_offset;/* offset to replacement instruction */ u16 cpuid;/* cpuid bit set for replacement */ u8 instrlen;/* length of original instruction */ u8 replacementlen;/* length of new instruction, <= instrlen */
@@ -84,8 +84,8 @@ static inline int alternatives_text_reserved(void *start, void *end)"661:\n\t" oldinstr"\n662:\n" \".section .altinstructions,\"a\"\n" \ _ASM_ALIGN"\n" \ - _ASM_PTR"661b\n" /* label */ \ - _ASM_PTR"663f\n" /* new instruction */ \ +" .long 661b - .\n" /* label */ \ +" .long 663f - .\n" /* new instruction */ \ " .word " __stringify(feature) "\n" /* feature bit */ \" .byte 662b-661b\n" /* sourcelen */ \" .byte 664f-663f\n" /* replacementlen */ \
arch/x86/include/asm/cpufeature.h +4 -4
@@ -331,8 +331,8 @@ static __always_inline __pure bool __static_cpu_has(u16 bit)"2:\n" ".section .altinstructions,\"a\"\n" _ASM_ALIGN"\n" - _ASM_PTR"1b\n" - _ASM_PTR"0\n" /* no replacement */ +" .long 1b - .\n" +" .long 0\n" /* no replacement */ " .word %P0\n" /* feature bit */ " .byte 2b - 1b\n" /* source len */ " .byte 0\n" /* replacement len */
@@ -349,8 +349,8 @@ static __always_inline __pure bool __static_cpu_has(u16 bit)"2:\n" ".section .altinstructions,\"a\"\n" _ASM_ALIGN"\n" - _ASM_PTR"1b\n" - _ASM_PTR"3f\n" +" .long 1b - .\n" +" .long 3f - .\n" " .word %P1\n" /* feature bit */ " .byte 2b - 1b\n" /* source len */ " .byte 4f - 3f\n" /* replacement len */
arch/x86/kernel/alternative.c +13 -8
@@ -263,6 +263,7 @@ void __init_or_module apply_alternatives(struct alt_instr *start, struct alt_instr *end) {struct alt_instr *a; +u8 *instr, *replacement;u8 insnbuf[MAX_PATCH_LEN ];DPRINTK ("%s: alt table %p -> %p\n" , __func__, start, end);
@@ -276,25 +277,29 @@ void __init_or_module apply_alternatives(struct alt_instr *start,* order. */ for (a = start; a < end; a++) { -u8 *instr = a->instr; + instr = (u8 *)&a->instr_offset + a->instr_offset; + replacement = (u8 *)&a->repl_offset + a->repl_offset;BUG_ON (a->replacementlen > a->instrlen);BUG_ON (a->instrlen >sizeof (insnbuf));BUG_ON (a->cpuid >=NCAPINTS *32 );if (!boot_cpu_has (a->cpuid))continue ; + +memcpy (insnbuf, replacement, a->replacementlen); + +/* 0xe8 is a relative jump; fix the offset. */ +if (*insnbuf ==0xe8 && a->replacementlen ==5 ) + *(s32 *)(insnbuf +1 ) += replacement - instr; + +add_nops (insnbuf + a->replacementlen, + a->instrlen - a->replacementlen); +#ifdef CONFIG_X86_64 /* vsyscall code is not mapped yet. resolve it manually. */ if (instr >= (u8 *)VSYSCALL_START && instr < (u8 *)VSYSCALL_END ) { instr =__va (instr - (u8 *)VSYSCALL_START + (u8 *)__pa_symbol (&__vsyscall_0)); -DPRINTK ("%s: vsyscall fixup: %p => %p\n" , - __func__, a->instr, instr); }#endif -memcpy (insnbuf, a->replacement, a->replacementlen); -if (*insnbuf ==0xe8 && a->replacementlen ==5 ) - *(s32 *)(insnbuf +1 ) += a->replacement - a->instr; -add_nops (insnbuf + a->replacementlen, - a->instrlen - a->replacementlen);text_poke_early (instr, insnbuf, a->instrlen); } }
arch/x86/lib/copy_page_64.S +3 -6
@@ -2,6 +2,7 @@ #include <linux/linkage.h> #include <asm/dwarf2.h> +#include <asm/alternative-asm.h> ALIGN copy_page_c:
@@ -110,10 +111,6 @@ ENDPROC(copy_page) 2: .previous .section .altinstructions,"a" - .align 8 - .quad copy_page - .quad 1b - .word X86_FEATURE_REP_GOOD - .byte .Lcopy_page_end - copy_page - .byte 2b - 1b + altinstruction_entry copy_page, 1b, X86_FEATURE_REP_GOOD, \ + .Lcopy_page_end-copy_page, 2b-1b .previous
arch/x86/lib/memmove_64.S +5 -6
@@ -9,6 +9,7 @@ #include <linux/linkage.h> #include <asm/dwarf2.h> #include <asm/cpufeature.h> +#include <asm/alternative-asm.h> #undef memmove
@@ -214,11 +215,9 @@ ENTRY(memmove) .previous .section .altinstructions,"a" - .align 8 - .quad .Lmemmove_begin_forward - .quad .Lmemmove_begin_forward_efs - .word X86_FEATURE_ERMS - .byte .Lmemmove_end_forward-.Lmemmove_begin_forward - .byte .Lmemmove_end_forward_efs-.Lmemmove_begin_forward_efs + altinstruction_entry .Lmemmove_begin_forward, \ + .Lmemmove_begin_forward_efs,X86_FEATURE_ERMS, \ + .Lmemmove_end_forward-.Lmemmove_begin_forward, \ + .Lmemmove_end_forward_efs-.Lmemmove_begin_forward_efs .previous ENDPROC(memmove)
Signature · 1 line
1.7.6
[tip:x86/vdso] x86: Make alternative instruction pointers relative
Commit-ID: 59e97e4d6fbcd5b74a94cb48bcbfc6f8478a5e93 Gitweb: http://git.kernel.org/tip/59e97e4d6fbcd5b74a94cb48bcbfc6f8478a5e93 Author: Andy Lutomirski <[email protected]> AuthorDate: Wed, 13 Jul 2011 09:24:10 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 13 Jul 2011 11:22:56 -0700
x86: Make alternative instruction pointers relative
This save a few bytes on x86-64 and means that future patches can apply alternatives to unrelocated code.
Signed-off-by Andy Lutomirski <[email protected]>
Link http://lkml.kernel.org/r/ff64a6b9a1a3860ca4a7b8b6dc7b4754f9491cd7.1310563276.git.luto@mit.edu open here
Signed-off-by H. Peter Anvin <[email protected]>
--- arch/x86/include/asm/alternative-asm.h | 4 ++-- arch/x86/include/asm/alternative.h | 8 ++++---- arch/x86/include/asm/cpufeature.h | 8 ++++---- arch/x86/kernel/alternative.c | 21 +++++++++++++-------- arch/x86/lib/copy_page_64.S | 9 +++------ arch/x86/lib/memmove_64.S | 11 +++++------ 6 files changed, 31 insertions(+), 30 deletions(-)
arch/x86/include/asm/alternative-asm.h +2 -2
@@ -17,8 +17,8 @@ .macro altinstruction_entry orig alt feature orig_len alt_len .align 8 - .quad \orig - .quad \alt + .long \orig - . + .long \alt - . .word \feature .byte \orig_len .byte \alt_len
arch/x86/include/asm/alternative.h +4 -4
@@ -43,8 +43,8 @@#endif struct alt_instr { -u8 *instr;/* original instruction */ -u8 *replacement; +s32 instr_offset;/* original instruction */ +s32 repl_offset;/* offset to replacement instruction */ u16 cpuid;/* cpuid bit set for replacement */ u8 instrlen;/* length of original instruction */ u8 replacementlen;/* length of new instruction, <= instrlen */
@@ -84,8 +84,8 @@ static inline int alternatives_text_reserved(void *start, void *end)"661:\n\t" oldinstr"\n662:\n" \".section .altinstructions,\"a\"\n" \ _ASM_ALIGN"\n" \ - _ASM_PTR"661b\n" /* label */ \ - _ASM_PTR"663f\n" /* new instruction */ \ +" .long 661b - .\n" /* label */ \ +" .long 663f - .\n" /* new instruction */ \ " .word " __stringify(feature) "\n" /* feature bit */ \" .byte 662b-661b\n" /* sourcelen */ \" .byte 664f-663f\n" /* replacementlen */ \
arch/x86/include/asm/cpufeature.h +4 -4
@@ -331,8 +331,8 @@ static __always_inline __pure bool __static_cpu_has(u16 bit)"2:\n" ".section .altinstructions,\"a\"\n" _ASM_ALIGN"\n" - _ASM_PTR"1b\n" - _ASM_PTR"0\n" /* no replacement */ +" .long 1b - .\n" +" .long 0\n" /* no replacement */ " .word %P0\n" /* feature bit */ " .byte 2b - 1b\n" /* source len */ " .byte 0\n" /* replacement len */
@@ -349,8 +349,8 @@ static __always_inline __pure bool __static_cpu_has(u16 bit)"2:\n" ".section .altinstructions,\"a\"\n" _ASM_ALIGN"\n" - _ASM_PTR"1b\n" - _ASM_PTR"3f\n" +" .long 1b - .\n" +" .long 3f - .\n" " .word %P1\n" /* feature bit */ " .byte 2b - 1b\n" /* source len */ " .byte 4f - 3f\n" /* replacement len */
arch/x86/kernel/alternative.c +13 -8
@@ -263,6 +263,7 @@ void __init_or_module apply_alternatives(struct alt_instr *start, struct alt_instr *end) {struct alt_instr *a; +u8 *instr, *replacement;u8 insnbuf[MAX_PATCH_LEN ];DPRINTK ("%s: alt table %p -> %p\n" , __func__, start, end);
@@ -276,25 +277,29 @@ void __init_or_module apply_alternatives(struct alt_instr *start,* order. */ for (a = start; a < end; a++) { -u8 *instr = a->instr; + instr = (u8 *)&a->instr_offset + a->instr_offset; + replacement = (u8 *)&a->repl_offset + a->repl_offset;BUG_ON (a->replacementlen > a->instrlen);BUG_ON (a->instrlen >sizeof (insnbuf));BUG_ON (a->cpuid >=NCAPINTS *32 );if (!boot_cpu_has (a->cpuid))continue ; + +memcpy (insnbuf, replacement, a->replacementlen); + +/* 0xe8 is a relative jump; fix the offset. */ +if (*insnbuf ==0xe8 && a->replacementlen ==5 ) + *(s32 *)(insnbuf +1 ) += replacement - instr; + +add_nops (insnbuf + a->replacementlen, + a->instrlen - a->replacementlen); +#ifdef CONFIG_X86_64 /* vsyscall code is not mapped yet. resolve it manually. */ if (instr >= (u8 *)VSYSCALL_START && instr < (u8 *)VSYSCALL_END ) { instr =__va (instr - (u8 *)VSYSCALL_START + (u8 *)__pa_symbol (&__vsyscall_0)); -DPRINTK ("%s: vsyscall fixup: %p => %p\n" , - __func__, a->instr, instr); }#endif -memcpy (insnbuf, a->replacement, a->replacementlen); -if (*insnbuf ==0xe8 && a->replacementlen ==5 ) - *(s32 *)(insnbuf +1 ) += a->replacement - a->instr; -add_nops (insnbuf + a->replacementlen, - a->instrlen - a->replacementlen);text_poke_early (instr, insnbuf, a->instrlen); } }
arch/x86/lib/copy_page_64.S +3 -6
@@ -2,6 +2,7 @@ #include <linux/linkage.h> #include <asm/dwarf2.h> +#include <asm/alternative-asm.h> ALIGN copy_page_c:
@@ -110,10 +111,6 @@ ENDPROC(copy_page) 2: .previous .section .altinstructions,"a" - .align 8 - .quad copy_page - .quad 1b - .word X86_FEATURE_REP_GOOD - .byte .Lcopy_page_end - copy_page - .byte 2b - 1b + altinstruction_entry copy_page, 1b, X86_FEATURE_REP_GOOD, \ + .Lcopy_page_end-copy_page, 2b-1b .previous
arch/x86/lib/memmove_64.S +5 -6
@@ -9,6 +9,7 @@ #include <linux/linkage.h> #include <asm/dwarf2.h> #include <asm/cpufeature.h> +#include <asm/alternative-asm.h> #undef memmove
@@ -214,11 +215,9 @@ ENTRY(memmove) .previous .section .altinstructions,"a" - .align 8 - .quad .Lmemmove_begin_forward - .quad .Lmemmove_begin_forward_efs - .word X86_FEATURE_ERMS - .byte .Lmemmove_end_forward-.Lmemmove_begin_forward - .byte .Lmemmove_end_forward_efs-.Lmemmove_begin_forward_efs + altinstruction_entry .Lmemmove_begin_forward, \ + .Lmemmove_begin_forward_efs,X86_FEATURE_ERMS, \ + .Lmemmove_end_forward-.Lmemmove_begin_forward, \ + .Lmemmove_end_forward_efs-.Lmemmove_begin_forward_efs .previous ENDPROC(memmove)
PATCH v3 3/8 x86-64: Allow alternative patching in the vDSO
In reply to Andy LutomirskiThis code is short enough and different enough from the module loader that it's not worth trying to share anything.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/vdso/vma.c | 33 +++++++++++++++++++++++++++++++++ 1 files changed, 33 insertions(+), 0 deletions(-)
arch/x86/vdso/vma.c +33 -0
@@ -23,11 +23,44 @@ extern unsigned short vdso_sync_cpuid;static struct page **vdso_pages;static unsigned vdso_size; +static void __init patch_vdso (void *vdso,size_t len) +{ +Elf64_Ehdr *hdr = vdso; +Elf64_Shdr *sechdrs, *alt_sec =0 ; +char *secstrings; +void *alt_data; +int i; + +BUG_ON (len <sizeof (Elf64_Ehdr)); +BUG_ON (memcmp (hdr->e_ident,ELFMAG ,SELFMAG ) !=0 ); + + sechdrs = (void *)hdr + hdr->e_shoff; + secstrings = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset; + +for (i =1 ; i < hdr->e_shnum; i++) { +Elf64_Shdr *shdr = &sechdrs[i]; +if (!strcmp (secstrings + shdr->sh_name,".altinstructions" )) { + alt_sec = shdr; + goto found; + } + } + +/* If we get here, it's probably a bug. */ +pr_warning ("patch_vdso: .altinstructions not found\n" ); +return ;/* nothing to patch */ + +found: + alt_data = (void *)hdr + alt_sec->sh_offset; +apply_alternatives (alt_data, alt_data + alt_sec->sh_size); +} +static int __init init_vdso_vars (void ) {int npages = (vdso_end - vdso_start +PAGE_SIZE -1 ) /PAGE_SIZE ;int i; +patch_vdso (vdso_start, vdso_end - vdso_start); + vdso_size = npages <<PAGE_SHIFT ; vdso_pages =kmalloc (sizeof (struct page *) * npages,GFP_KERNEL );if (!vdso_pages)
Signature · 1 line
1.7.6
[tip:x86/vdso] x86-64: Allow alternative patching in the vDSO
Commit-ID: 1b3f2a72bbcfdf92e368a44448c45eb639b05b5e Gitweb: http://git.kernel.org/tip/1b3f2a72bbcfdf92e368a44448c45eb639b05b5e Author: Andy Lutomirski <[email protected]> AuthorDate: Wed, 13 Jul 2011 09:24:11 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 13 Jul 2011 11:23:07 -0700
x86-64: Allow alternative patching in the vDSO
This code is short enough and different enough from the module loader that it's not worth trying to share anything.
Signed-off-by Andy Lutomirski <[email protected]>
Link http://lkml.kernel.org/r/e73112e4381fff29e31b882c2d0856822edaea53.1310563276.git.luto@mit.edu open here
Signed-off-by H. Peter Anvin <[email protected]>
--- arch/x86/vdso/vma.c | 33 +++++++++++++++++++++++++++++++++ 1 files changed, 33 insertions(+), 0 deletions(-)
arch/x86/vdso/vma.c +33 -0
@@ -23,11 +23,44 @@ extern unsigned short vdso_sync_cpuid;static struct page **vdso_pages;static unsigned vdso_size; +static void __init patch_vdso (void *vdso,size_t len) +{ +Elf64_Ehdr *hdr = vdso; +Elf64_Shdr *sechdrs, *alt_sec =0 ; +char *secstrings; +void *alt_data; +int i; + +BUG_ON (len <sizeof (Elf64_Ehdr)); +BUG_ON (memcmp (hdr->e_ident,ELFMAG ,SELFMAG ) !=0 ); + + sechdrs = (void *)hdr + hdr->e_shoff; + secstrings = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset; + +for (i =1 ; i < hdr->e_shnum; i++) { +Elf64_Shdr *shdr = &sechdrs[i]; +if (!strcmp (secstrings + shdr->sh_name,".altinstructions" )) { + alt_sec = shdr; + goto found; + } + } + +/* If we get here, it's probably a bug. */ +pr_warning ("patch_vdso: .altinstructions not found\n" ); +return ;/* nothing to patch */ + +found: + alt_data = (void *)hdr + alt_sec->sh_offset; +apply_alternatives (alt_data, alt_data + alt_sec->sh_size); +} +static int __init init_vdso_vars (void ) {int npages = (vdso_end - vdso_start +PAGE_SIZE -1 ) /PAGE_SIZE ;int i; +patch_vdso (vdso_start, vdso_end - vdso_start); + vdso_size = npages <<PAGE_SHIFT ; vdso_pages =kmalloc (sizeof (struct page *) * npages,GFP_KERNEL );if (!vdso_pages)
Borislav Petkov
maintainerRe: [tip:x86/vdso] x86-64: Allow alternative patching in the vDSO
Quoted 7 lines · tip-bot for Andy Lutomirski
Commit-ID: 1b3f2a72bbcfdf92e368a44448c45eb639b05b5e Gitweb: http://git.kernel.org/tip/1b3f2a72bbcfdf92e368a44448c45eb639b05b5e Author: Andy Lutomirski <[email protected]> AuthorDate: Wed, 13 Jul 2011 09:24:11 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 13 Jul 2011 11:23:07 -0700
Quoted 8 lines
x86-64: Allow alternative patching in the vDSO
This code is short enough and different enough from the module loader that it's not worth trying to share anything.
--
From: Borislav Petkov <[email protected]>
Date: Mon, 18 Jul 2011 21:07:25 +0200
Subject: [PATCH] x86, vdso: Drop now wrong comment
Now that 1b3f2a72bbcfdf92e368a44448c45eb639b05b5e is in, it is very important that the below lying comment be removed! :-)
Signed-off-by Borislav Petkov <[email protected]>
--- arch/x86/vdso/vclock_gettime.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-)
arch/x86/vdso/vclock_gettime.c +0 -1
@@ -6,7 +6,6 @@* * The code should have no internal unresolved relocations. * Check with readelf after changing. -* Also alternative() doesn't work. */ /* Disable profiling for userspace code: */
Signature · 5 lines
1.7.5.3.401.gfb674
--
Regards/Gruss,
Boris.Andrew Lutomirski
Acked-byRe: [tip:x86/vdso] x86-64: Allow alternative patching in the vDSO
Quoted 35 lines · Borislav Petkov
Quoted 7 lines · tip-bot for Andy Lutomirski
Commit-ID: 1b3f2a72bbcfdf92e368a44448c45eb639b05b5e
Gitweb: http://git.kernel.org/tip/1b3f2a72bbcfdf92e368a44448c45eb639b05b5e
Author: Andy Lutomirski <[email protected]>
AuthorDate: Wed, 13 Jul 2011 09:24:11 -0400
Committer: H. Peter Anvin <[email protected]>
CommitDate: Wed, 13 Jul 2011 11:23:07 -0700Quoted 8 lines
x86-64: Allow alternative patching in the vDSO
This code is short enough and different enough from the module loader that it's not worth trying to share anything.
--
From: Borislav Petkov <[email protected]>
Date: Mon, 18 Jul 2011 21:07:25 +0200
Subject: [PATCH] x86, vdso: Drop now wrong commentNow that
1b3f2a72bbcfdf92e368a44448c45eb639b05b5eis in, it is very important that the below lying comment be removed! :-)Signed-off-by Borislav Petkov <[email protected]>---
arch/x86/vdso/vclock_gettime.c | 1 -1 files changed, 0 insertions(+), 1 deletions(-)arch/x86/vdso/vclock_gettime.c +0
-0
arch/x86/vdso/vclock_gettime.c line 9
Quoted 5 lines
arch/x86/vdso/vclock_gettime.c +0
-1@@ -6,7 +6,6 @@* * The code should have no internal unresolved relocations. * Check with readelf after changing. -* Also alternative() doesn't work. */
That was the whole point, after all.
Acked-by Andy Lutomirski <[email protected]>
Quoted 9 lines
arch/x86/vdso/vclock_gettime.c +0
-1/* Disable profiling for userspace code: */ --1.7.5.3.401.gfb674
--
Regards/Gruss,
Boris.
tip-bot for Borislav Petkov
maintainer[tip:x86/vdso] x86, vdso: Drop now wrong comment
In reply to Borislav PetkovCommit-ID: 8c400f6ce068366bc3517f1036bb99169cfec9cd Gitweb: http://git.kernel.org/tip/8c400f6ce068366bc3517f1036bb99169cfec9cd Author: Borislav Petkov <[email protected]> AuthorDate: Mon, 18 Jul 2011 21:10:54 +0200 Committer: H. Peter Anvin <[email protected]> CommitDate: Mon, 18 Jul 2011 12:29:50 -0700
x86, vdso: Drop now wrong comment
Now that 1b3f2a72bbcfdf92e368a44448c45eb639b05b5e is in, it is very important that the below lying comment be removed! :-)
Signed-off-by Borislav Petkov <[email protected]>
Acked-by Andy Lutomirski <[email protected]>
Signed-off-by H. Peter Anvin <[email protected]>
--- arch/x86/vdso/vclock_gettime.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-)
arch/x86/vdso/vclock_gettime.c +0 -1
@@ -6,7 +6,6 @@* * The code should have no internal unresolved relocations. * Check with readelf after changing. -* Also alternative() doesn't work. */ /* Disable profiling for userspace code: */
PATCH v3 4/8 x86-64: Add --no-undefined to vDSO build
In reply to Andy LutomirskiThis gives much nicer diagnostics when something goes wrong. It's supported at least as far back as binutils 2.15.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/vdso/Makefile | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
arch/x86/vdso/Makefile +1 -0
@@ -26,6 +26,7 @@ targets += vdso.so vdso.so.dbg vdso.lds $(vobjs-y) export CPPFLAGS_vdso.lds += -P -C VDSO_LDFLAGS_vdso.lds = -m64 -Wl,-soname=linux-vdso.so.1 \ + -Wl,--no-undefined \ -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096 $(obj)/vdso.o: $(src)/vdso.S $(obj)/vdso.so
Signature · 1 line
1.7.6
[tip:x86/vdso] x86-64: Add --no-undefined to vDSO build
Commit-ID: 7f79ad15f33cf4968cafb0e3d2beba427de01d3a Gitweb: http://git.kernel.org/tip/7f79ad15f33cf4968cafb0e3d2beba427de01d3a Author: Andy Lutomirski <[email protected]> AuthorDate: Wed, 13 Jul 2011 09:24:12 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 13 Jul 2011 11:23:09 -0700
x86-64: Add --no-undefined to vDSO build
This gives much nicer diagnostics when something goes wrong. It's supported at least as far back as binutils 2.15.
Signed-off-by Andy Lutomirski <[email protected]>
Link http://lkml.kernel.org/r/de0b50920469ff6359c529526e7639fdd36fa83c.1310563276.git.luto@mit.edu open here
Signed-off-by H. Peter Anvin <[email protected]>
--- arch/x86/vdso/Makefile | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
arch/x86/vdso/Makefile +1 -0
@@ -26,6 +26,7 @@ targets += vdso.so vdso.so.dbg vdso.lds $(vobjs-y) export CPPFLAGS_vdso.lds += -P -C VDSO_LDFLAGS_vdso.lds = -m64 -Wl,-soname=linux-vdso.so.1 \ + -Wl,--no-undefined \ -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096 $(obj)/vdso.o: $(src)/vdso.S $(obj)/vdso.so
PATCH v3 5/8 clocksource: Replace vread with generic arch data
In reply to Andy LutomirskiThe vread field was bloating struct clocksource everywhere except x86_64, and I want to change the way this works on x86_64, so let's split it out into per-arch data.
Cc Clemens Ladisch <[email protected]>
Cc Tony Luck <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc John Stultz <[email protected]>
Cc Thomas Gleixner <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/include/asm/clocksource.h | 16 ++++++++++++++++ arch/x86/kernel/hpet.c | 2 +- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vsyscall_64.c | 2 +- include/asm-generic/clocksource.h | 4 ++++ include/linux/clocksource.h | 10 ++++++++-- 6 files changed, 31 insertions(+), 5 deletions(-) create mode 100644 arch/x86/include/asm/clocksource.h create mode 100644 include/asm-generic/clocksource.h
arch/x86/include/asm/clocksource.h new file +16 -0
@@ -0,0 +1,16 @@ +/* x86-specific clocksource additions */ + +#ifndef _ASM_X86_CLOCKSOURCE_H +#define _ASM_X86_CLOCKSOURCE_H + +#ifdef CONFIG_X86_64 + +#define __ARCH_HAS_CLOCKSOURCE_DATA + +struct arch_clocksource_data { +cycle_t (*vread)(void ); +}; + +#endif /* CONFIG_X86_64 */ + +#endif /* _ASM_X86_CLOCKSOURCE_H */
arch/x86/kernel/hpet.c +1 -1
@@ -753,7 +753,7 @@ static struct clocksource clocksource_hpet = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .resume = hpet_resume_counter, #ifdef CONFIG_X86_64 - .vread = vread_hpet, + .archdata = { .vread = vread_hpet },#endif };
arch/x86/kernel/tsc.c +1 -1
@@ -777,7 +777,7 @@ static struct clocksource clocksource_tsc = { .flags = CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_MUST_VERIFY, #ifdef CONFIG_X86_64 - .vread = vread_tsc, + .archdata = { .vread = vread_tsc },#endif };
arch/x86/kernel/vsyscall_64.c +1 -1
@@ -74,7 +74,7 @@ void update_vsyscall(struct timespec *wall_time, struct timespec *wtm,write_seqlock_irqsave (&vsyscall_gtod_data.lock, flags);/* copy vsyscall data */ - vsyscall_gtod_data.clock.vread = clock->vread; + vsyscall_gtod_data.clock.vread = clock->archdata.vread; vsyscall_gtod_data.clock.cycle_last = clock->cycle_last; vsyscall_gtod_data.clock.mask = clock->mask; vsyscall_gtod_data.clock.mult = mult;
include/asm-generic/clocksource.h new file +4 -0
@@ -0,0 +1,4 @@ +/* +* Architectures should override this file to add private userspace +* clock magic if needed. +*/
include/linux/clocksource.h +8 -2
@@ -22,6 +22,8 @@typedef u64 cycle_t ;struct clocksource ; +#include <asm/clocksource.h> +/** * struct cyclecounter - hardware abstraction for a free running counter * Provides completely state-free accessors to the underlying hardware.
@@ -153,7 +155,7 @@ extern u64 timecounter_cyc2time(struct timecounter *tc,* @shift: cycle to nanosecond divisor (power of two) * @max_idle_ns: max idle time permitted by the clocksource (nsecs) * @flags: flags describing special properties -* @vread: vsyscall based read +* @archdata: arch-specific data * @suspend: suspend function for the clocksource, if necessary * @resume: resume function for the clocksource, if necessary */
@@ -175,10 +177,14 @@ struct clocksource {#else #define CLKSRC_FSYS_MMIO_SET (mmio, addr) do { } while (0)#endif + +#ifdef __ARCH_HAS_CLOCKSOURCE_DATA +struct arch_clocksource_data archdata; +#endif +const char *name;struct list_head list;int rating; -cycle_t (*vread)(void);int (*enable)(struct clocksource *cs);void (*disable)(struct clocksource *cs);unsigned long flags;
Signature · 1 line
1.7.6
[tip:x86/vdso] clocksource: Replace vread with generic arch data
Commit-ID: 433bd805e5fd2c731b3a9025b034f066272d336e Gitweb: http://git.kernel.org/tip/433bd805e5fd2c731b3a9025b034f066272d336e Author: Andy Lutomirski <[email protected]> AuthorDate: Wed, 13 Jul 2011 09:24:13 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 13 Jul 2011 11:23:12 -0700
clocksource: Replace vread with generic arch data
The vread field was bloating struct clocksource everywhere except x86_64, and I want to change the way this works on x86_64, so let's split it out into per-arch data.
Cc Clemens Ladisch <[email protected]>
Cc Tony Luck <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc John Stultz <[email protected]>
Cc Thomas Gleixner <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
Link http://lkml.kernel.org/r/3ae5ec76a168eaaae63f08a2a1060b91aa0b7759.1310563276.git.luto@mit.edu open here
Signed-off-by H. Peter Anvin <[email protected]>
--- arch/x86/include/asm/clocksource.h | 16 ++++++++++++++++ arch/x86/kernel/hpet.c | 2 +- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vsyscall_64.c | 2 +- include/asm-generic/clocksource.h | 4 ++++ include/linux/clocksource.h | 10 ++++++++-- 6 files changed, 31 insertions(+), 5 deletions(-)
arch/x86/include/asm/clocksource.h new file +16 -0
@@ -0,0 +1,16 @@ +/* x86-specific clocksource additions */ + +#ifndef _ASM_X86_CLOCKSOURCE_H +#define _ASM_X86_CLOCKSOURCE_H + +#ifdef CONFIG_X86_64 + +#define __ARCH_HAS_CLOCKSOURCE_DATA + +struct arch_clocksource_data { +cycle_t (*vread)(void ); +}; + +#endif /* CONFIG_X86_64 */ + +#endif /* _ASM_X86_CLOCKSOURCE_H */
arch/x86/kernel/hpet.c +1 -1
@@ -753,7 +753,7 @@ static struct clocksource clocksource_hpet = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .resume = hpet_resume_counter, #ifdef CONFIG_X86_64 - .vread = vread_hpet, + .archdata = { .vread = vread_hpet },#endif };
arch/x86/kernel/tsc.c +1 -1
@@ -777,7 +777,7 @@ static struct clocksource clocksource_tsc = { .flags = CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_MUST_VERIFY, #ifdef CONFIG_X86_64 - .vread = vread_tsc, + .archdata = { .vread = vread_tsc },#endif };
arch/x86/kernel/vsyscall_64.c +1 -1
@@ -74,7 +74,7 @@ void update_vsyscall(struct timespec *wall_time, struct timespec *wtm,write_seqlock_irqsave (&vsyscall_gtod_data.lock, flags);/* copy vsyscall data */ - vsyscall_gtod_data.clock.vread = clock->vread; + vsyscall_gtod_data.clock.vread = clock->archdata.vread; vsyscall_gtod_data.clock.cycle_last = clock->cycle_last; vsyscall_gtod_data.clock.mask = clock->mask; vsyscall_gtod_data.clock.mult = mult;
include/asm-generic/clocksource.h new file +4 -0
@@ -0,0 +1,4 @@ +/* +* Architectures should override this file to add private userspace +* clock magic if needed. +*/
include/linux/clocksource.h +8 -2
@@ -22,6 +22,8 @@typedef u64 cycle_t ;struct clocksource ; +#include <asm/clocksource.h> +/** * struct cyclecounter - hardware abstraction for a free running counter * Provides completely state-free accessors to the underlying hardware.
@@ -153,7 +155,7 @@ extern u64 timecounter_cyc2time(struct timecounter *tc,* @shift: cycle to nanosecond divisor (power of two) * @max_idle_ns: max idle time permitted by the clocksource (nsecs) * @flags: flags describing special properties -* @vread: vsyscall based read +* @archdata: arch-specific data * @suspend: suspend function for the clocksource, if necessary * @resume: resume function for the clocksource, if necessary */
@@ -175,10 +177,14 @@ struct clocksource {#else #define CLKSRC_FSYS_MMIO_SET (mmio, addr) do { } while (0)#endif + +#ifdef __ARCH_HAS_CLOCKSOURCE_DATA +struct arch_clocksource_data archdata; +#endif +const char *name;struct list_head list;int rating; -cycle_t (*vread)(void);int (*enable)(struct clocksource *cs);void (*disable)(struct clocksource *cs);unsigned long flags;
H. Peter Anvin
reviewerRe: [tip:x86/vdso] clocksource: Replace vread with generic arch data
Quoted 35 lines · tip-bot for Andy Lutomirski
arch/x86/include/asm/clocksource.h new file +16
-0@@ -0,0 +1,16 @@ +/* x86-specific clocksource additions */ + +#ifndef _ASM_X86_CLOCKSOURCE_H +#define _ASM_X86_CLOCKSOURCE_H + +#ifdef CONFIG_X86_64 + +#define __ARCH_HAS_CLOCKSOURCE_DATA + +struct arch_clocksource_data { +cycle_t (*vread)(void ); +}; + +#endif /* CONFIG_X86_64 */ + +#endif /* _ASM_X86_CLOCKSOURCE_H */ include/asm-generic/clocksource.h new file +4
-0@@ -0,0 +1,4 @@ +/* +* Architectures should override this file to add private userspace +* clock magic if needed. +*/ include/linux/clocksource.h +0
-0
include/linux/clocksource.h line 27
Quoted 6 lines
include/linux/clocksource.h +2
-0@@ -22,6 +22,8 @@typedef u64 cycle_t ;struct clocksource ; +#include <asm/clocksource.h> +/**
Hi Andy,
I should have spotted this sooner... Ingo pointed out to me that this breaks building on any non-x86 architecture.
asm-generic doesn't work quite the way you think it does, here; it's a library for architectures to include from, not something that gets included on all architectures by default.
To make a file from asm-generic appear in asm/ it needs to at least appear in a generic-y statement in a Makefile; however, that is kind of pointless in the case of an empty file.
One could argue that it would be nice if we had such a fallback director, of if asm-generic was such a fallback directory, but currently it is not.
The easiest way to deal with this is probably to make ARCH_HAS_CLOCKSOURCE_DATA here a Kconfig option (autoselected for x86-64); the only other would be to add this as generic-y stubs for every single architecture.
Cc: Arnd Bergmann who is the asm-generic maintainer for a recommendation, and linux-arch.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Re: [tip:x86/vdso] clocksource: Replace vread with generic arch data
Quoted 45 lines · H. Peter Anvin
Quoted 35 lines · tip-bot for Andy Lutomirski
arch/x86/include/asm/clocksource.h new file +16
-0@@ -0,0 +1,16 @@ +/* x86-specific clocksource additions */ + +#ifndef _ASM_X86_CLOCKSOURCE_H +#define _ASM_X86_CLOCKSOURCE_H + +#ifdef CONFIG_X86_64 + +#define __ARCH_HAS_CLOCKSOURCE_DATA + +struct arch_clocksource_data { + cycle_t (*vread)(void); +}; + +#endif /* CONFIG_X86_64 */ + +#endif /* _ASM_X86_CLOCKSOURCE_H */ include/asm-generic/clocksource.h new file +4
-0@@ -0,0 +1,4 @@ +/* +* Architectures should override this file to add private userspace +* clock magic if needed. +*/ include/linux/clocksource.h +0
-0include/linux/clocksource.h line 27
Quoted 6 lines
include/linux/clocksource.h +2
-0@@ -22,6 +22,8 @@ typedef u64 cycle_t; struct clocksource; +#include <asm/clocksource.h> + /**Hi Andy,
Quoted 6 lines
I should have spotted this sooner... Ingo pointed out to me that this breaks building on any non-x86 architecture.
asm-generic doesn't work quite the way you think it does, here; it's a library for architectures to include from, not something that gets included on all architectures by default.
Whoops. If only cross-compiler toolchains were easy to build...
[...]
Quoted 8 lines
The easiest way to deal with this is probably to make
ARCH_HAS_CLOCKSOURCE_DATAhere aKconfigoption (autoselected for x86-64); the only other would be to add this as generic-y stubs for every single architecture.Cc: Arnd Bergmann who is the asm-generic maintainer for a recommendation, and linux-arch.
ARCH_HAS_CLOCKSOURCE_DATA seems reasonable. It's a little ugly because it needs:
#ifdef CONFIG_ARCH_HAS_CLOCKSOURCE_DATA
#include <asm/clocksource.h>
#endif
If I don't hear any better suggestions, I'll implement that tomorrow. Do you want an incremental patch or a replacement?
--Andy
H. Peter Anvin
reviewerRe: [tip:x86/vdso] clocksource: Replace vread with generic arch data
Quoted 3 lines · Andrew Lutomirski
Whoops. If only cross-compiler toolchains were easy to build...
http://www.kernel.org/pub/tools/crosstool/
Quoted 4 lines
If I don't hear any better suggestions, I'll implement that tomorrow. Do you want an incremental patch or a replacement?
Already did, given the timing.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Re: [tip:x86/vdso] clocksource: Replace vread with generic arch data
Quoted 6 lines · H. Peter Anvin
Quoted 3 lines · Andrew Lutomirski
Whoops. If only cross-compiler toolchains were easy to build...
I didn't realize that thing was still maintained. Neat!
Quoted 7 lines
Quoted 5 lines
If I don't hear any better suggestions, I'll implement that tomorrow. Do you want an incremental patch or a replacement?
Already did, given the timing.
Presumably this needs a matching fix:
(Also, none of the IA-64 maintainers have replied to any of my requests for review.)
--Andy
H. Peter Anvin
reviewerRe: [tip:x86/vdso] clocksource: Replace vread with generic arch data
Quoted 8 lines · Andrew Lutomirski
Presumably this needs a matching fix:
(Also, none of the IA-64 maintainers have replied to any of my requests for review.)
What do you mean, "matching fix"? The fix I posted should work on x86 and ia64.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Re: [tip:x86/vdso] clocksource: Replace vread with generic arch data
Quoted 4 lines · H. Peter Anvin
Quoted 3 lines · Andrew Lutomirski
Presumably this needs a matching fix:
Quoted 9 lines
Quoted 5 lines
(Also, none of the IA-64 maintainers have replied to any of my requests for review.)
What do you mean, "matching fix"? The fix I posted should work on x86 and ia64.
*sigh* I read it wrong. Never mind.
--Andy
H. Peter Anvin
reviewerRe: [tip:x86/vdso] clocksource: Replace vread with generic arch data
Quoted 7 lines · Andrew Lutomirski
Quoted 4 lines
What do you mean, "matching fix"? The fix I posted should work on x86 and ia64.
*sigh* I read it wrong. Never mind.
That's ok, better to say something and be wrong than to not say something and be right...
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
tip-bot for H. Peter Anvin
reviewer[tip:x86/vdso] clocksource: Change __ARCH_HAS_CLOCKSOURCE_DATA to a CONFIG option
In reply to H. Peter AnvinCommit-ID: ae7bd11b471931752e5609094ca0a49386590524 Gitweb: http://git.kernel.org/tip/ae7bd11b471931752e5609094ca0a49386590524 Author: H. Peter Anvin <[email protected]> AuthorDate: Thu, 21 Jul 2011 13:34:05 -0700 Committer: H. Peter Anvin <[email protected]> CommitDate: Thu, 21 Jul 2011 13:34:05 -0700
clocksource: Change __ARCH_HAS_CLOCKSOURCE_DATA to a CONFIG option
The machinery for __ARCH_HAS_CLOCKSOURCE_DATA assumed a file in asm-generic would be the default for architectures without their own file in asm/, but that is not how it works.
Replace it with a Kconfig option instead.
Signed-off-by H. Peter Anvin <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Arnd Bergmann <[email protected]>
Cc Tony Luck <[email protected]>
--- arch/ia64/Kconfig | 3 +++ arch/ia64/include/asm/clocksource.h | 2 -- arch/x86/Kconfig | 4 ++++ arch/x86/include/asm/clocksource.h | 2 -- include/asm-generic/clocksource.h | 4 ---- include/linux/clocksource.h | 4 +++- 6 files changed, 10 insertions(+), 9 deletions(-)
arch/ia64/Kconfig +3 -0
@@ -101,6 +101,9 @@ config GENERIC_IOMAP bool default y +config ARCH_CLOCKSOURCE_DATA + def_bool y + config SCHED_OMIT_FRAME_POINTER bool default y
arch/ia64/include/asm/clocksource.h +0 -2
@@ -3,8 +3,6 @@#ifndef _ASM_IA64_CLOCKSOURCE_H#define _ASM_IA64_CLOCKSOURCE_H -#define __ARCH_HAS_CLOCKSOURCE_DATA -struct arch_clocksource_data {void *fsys_mmio;/* used by fsyscall asm code */ };
arch/x86/Kconfig +4 -0
@@ -93,6 +93,10 @@ config CLOCKSOURCE_WATCHDOG config GENERIC_CLOCKEVENTS def_bool y +config ARCH_CLOCKSOURCE_DATA + def_bool y + depends on X86_64 + config GENERIC_CLOCKEVENTS_BROADCAST def_bool y depends on X86_64 || (X86_32 && X86_LOCAL_APIC)
arch/x86/include/asm/clocksource.h +0 -2
@@ -5,8 +5,6 @@#ifdef CONFIG_X86_64 -#define __ARCH_HAS_CLOCKSOURCE_DATA -#define VCLOCK_NONE 0/* No vDSO clock available. */ #define VCLOCK_TSC 1/* vDSO should use vread_tsc. */ #define VCLOCK_HPET 2/* vDSO should use vread_hpet. */
include/asm-generic/clocksource.h deleted +0 -4
@@ -1,4 +0,0 @@ -/* -* Architectures should override this file to add private userspace -* clock magic if needed. -*/
include/linux/clocksource.h +3 -1
@@ -22,7 +22,9 @@typedef u64 cycle_t ;struct clocksource ; +#ifdef CONFIG_ARCH_CLOCKSOURCE_DATA #include <asm/clocksource.h> +#endif /** * struct cyclecounter - hardware abstraction for a free running counter
@@ -171,7 +173,7 @@ struct clocksource {u32 shift;u64 max_idle_ns; -#ifdef __ARCH_HAS_CLOCKSOURCE_DATA +#ifdef CONFIG_ARCH_CLOCKSOURCE_DATA struct arch_clocksource_data archdata;#endif
PATCH v3 6/8 x86-64: Move vread_tsc and vread_hpet into the vDSO
In reply to Andy LutomirskiThe vsyscall page now consists entirely of trap instructions.
Cc John Stultz <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/include/asm/clocksource.h | 6 +++- arch/x86/include/asm/tsc.h | 4 --- arch/x86/include/asm/vgtod.h | 2 +- arch/x86/include/asm/vsyscall.h | 4 --- arch/x86/kernel/Makefile | 7 +---- arch/x86/kernel/alternative.c | 8 ----- arch/x86/kernel/hpet.c | 9 +----- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 3 -- arch/x86/kernel/vread_tsc_64.c | 36 ------------------------- arch/x86/kernel/vsyscall_64.c | 2 +- arch/x86/vdso/vclock_gettime.c | 52 +++++++++++++++++++++++++++++++---- 12 files changed, 56 insertions(+), 79 deletions(-) delete mode 100644 arch/x86/kernel/vread_tsc_64.c
arch/x86/include/asm/clocksource.h +5 -1
@@ -7,8 +7,12 @@#define __ARCH_HAS_CLOCKSOURCE_DATA +#define VCLOCK_NONE 0/* No vDSO clock available. */ +#define VCLOCK_TSC 1/* vDSO should use vread_tsc. */ +#define VCLOCK_HPET 2/* vDSO should use vread_hpet. */ +struct arch_clocksource_data { -cycle_t (*vread)(void ); +int vclock_mode; };#endif /* CONFIG_X86_64 */
arch/x86/include/asm/tsc.h +0 -4
@@ -51,10 +51,6 @@ extern int unsynchronized_tsc(void);extern int check_tsc_unstable (void );extern unsigned long native_calibrate_tsc (void ); -#ifdef CONFIG_X86_64 -extern cycles_t vread_tsc (void ); -#endif -/* * Boot-time check whether the TSCs are synchronized across * all CPUs/cores:
arch/x86/include/asm/vgtod.h +1 -1
@@ -13,7 +13,7 @@ struct vsyscall_gtod_data {struct timezone sys_tz;struct {/* extract of a clocksource struct */ -cycle_t (*vread)(void ); +int vclock_mode;cycle_t cycle_last;cycle_t mask;u32 mult;
arch/x86/include/asm/vsyscall.h +0 -4
@@ -16,10 +16,6 @@ enum vsyscall_num {#ifdef __KERNEL__#include <linux/seqlock.h> -/* Definitions for CONFIG_GENERIC_TIME definitions */ -#define __vsyscall_fn \ - __attribute__ ((unused, __section__(".vsyscall_fn"))) notrace -#define VGETCPU_RDTSCP 1#define VGETCPU_LSL 2
arch/x86/kernel/Makefile +1 -6
@@ -24,17 +24,12 @@ endif nostackp := $(call cc-option, -fno-stack-protector) CFLAGS_vsyscall_64.o := $(PROFILING) -g0 $(nostackp) CFLAGS_hpet.o := $(nostackp) -CFLAGS_vread_tsc_64.o := $(nostackp) CFLAGS_paravirt.o := $(nostackp) GCOV_PROFILE_vsyscall_64.o := n GCOV_PROFILE_hpet.o := n GCOV_PROFILE_tsc.o := n -GCOV_PROFILE_vread_tsc_64.o := n GCOV_PROFILE_paravirt.o := n -# vread_tsc_64 is hot and should be fully optimized: -CFLAGS_REMOVE_vread_tsc_64.o = -pg -fno-optimize-sibling-calls - obj-y := process_$(BITS).o signal.o entry_$(BITS).o obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o ldt.o dumpstack.o
@@ -43,7 +38,7 @@ obj-$(CONFIG_IRQ_WORK) += irq_work.o obj-y += probe_roms.o obj-$(CONFIG_X86_32) += sys_i386_32.o i386_ksyms_32.o obj-$(CONFIG_X86_64) += sys_x86_64.o x8664_ksyms_64.o -obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o vread_tsc_64.o +obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o obj-$(CONFIG_X86_64) += vsyscall_emu_64.o obj-y += bootflag.o e820.o obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
arch/x86/kernel/alternative.c +0 -8
@@ -14,7 +14,6 @@#include <asm/pgtable.h> #include <asm/mce.h> #include <asm/nmi.h> -#include <asm/vsyscall.h> #include <asm/cacheflush.h> #include <asm/tlbflush.h> #include <asm/io.h>
@@ -250,7 +249,6 @@ static void __init_or_module add_nops(void *insns, unsigned int len)extern struct alt_instr __alt_instructions[], __alt_instructions_end[];extern s32 __smp_locks[], __smp_locks_end[]; -extern char __vsyscall_0;void *text_poke_early (void *addr,const void *opcode,size_t len);/* Replace instructions with better alternatives for this CPU type.
@@ -294,12 +292,6 @@ void __init_or_module apply_alternatives(struct alt_instr *start,add_nops (insnbuf + a->replacementlen, a->instrlen - a->replacementlen); -#ifdef CONFIG_X86_64 -/* vsyscall code is not mapped yet. resolve it manually. */ -if (instr >= (u8 *)VSYSCALL_START && instr < (u8 *)VSYSCALL_END ) { - instr =__va (instr - (u8 *)VSYSCALL_START + (u8 *)__pa_symbol (&__vsyscall_0)); - } -#endif text_poke_early (instr, insnbuf, a->instrlen); } }
arch/x86/kernel/hpet.c +1 -8
@@ -738,13 +738,6 @@ static cycle_t read_hpet(struct clocksource *cs)return (cycle_t )hpet_readl (HPET_COUNTER ); } -#ifdef CONFIG_X86_64 -static cycle_t __vsyscall_fn vread_hpet(void) -{ -return readl ((const void __iomem *)fix_to_virt (VSYSCALL_HPET ) +0xf0 ); -} -#endif -static struct clocksource clocksource_hpet = { .name ="hpet" , .rating =250 ,
@@ -753,7 +746,7 @@ static struct clocksource clocksource_hpet = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .resume = hpet_resume_counter, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_hpet }, + .archdata = { .vclock_mode = VCLOCK_HPET },#endif };
arch/x86/kernel/tsc.c +1 -1
@@ -777,7 +777,7 @@ static struct clocksource clocksource_tsc = { .flags = CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_MUST_VERIFY, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_tsc }, + .archdata = { .vclock_mode = VCLOCK_TSC },#endif };
arch/x86/kernel/vmlinux.lds.S +0 -3
@@ -169,9 +169,6 @@ SECTIONS .vsyscall : AT(VLOAD(.vsyscall)) { *(.vsyscall_0) - . = ALIGN(L1_CACHE_BYTES); - *(.vsyscall_fn) - . = 1024; *(.vsyscall_1)
arch/x86/kernel/vread_tsc_64.c deleted +0 -36
@@ -1,36 +0,0 @@ -/* This code runs in userspace. */ - -#define DISABLE_BRANCH_PROFILING -#include <asm/vgtod.h> - -notrace cycle_t __vsyscall_fn vread_tsc(void) -{ -cycle_t ret; -u64 last; - -/* -* Empirically, a fence (of type that depends on the CPU) -* before rdtsc is enough to ensure that rdtsc is ordered -* with respect to loads. The various CPU manuals are unclear -* as to whether rdtsc can be reordered with later loads, -* but no one has ever seen it happen. -*/ -rdtsc_barrier (); - ret = (cycle_t )vget_cycles (); - - last =VVAR (vsyscall_gtod_data).clock.cycle_last; - -if (likely (ret >= last)) -return ret; - -/* -* GCC likes to generate cmov here, but this branch is extremely -* predictable (it's just a funciton of time and the likely is -* very likely) and there's a data dependence, so force GCC -* to generate a branch instead. I don't barrier() because -* we don't actually need a barrier, and if this function -* ever gets inlined it will generate worse code. -*/ - asmvolatile ("" ); -return last; -}
arch/x86/kernel/vsyscall_64.c +1 -1
@@ -74,7 +74,7 @@ void update_vsyscall(struct timespec *wall_time, struct timespec *wtm,write_seqlock_irqsave (&vsyscall_gtod_data.lock, flags);/* copy vsyscall data */ - vsyscall_gtod_data.clock.vread = clock->archdata.vread; + vsyscall_gtod_data.clock.vclock_mode = clock->archdata.vclock_mode; vsyscall_gtod_data.clock.cycle_last = clock->cycle_last; vsyscall_gtod_data.clock.mask = clock->mask; vsyscall_gtod_data.clock.mult = mult;
arch/x86/vdso/vclock_gettime.c +46 -6
79 lines not on this page · raw message
Signature · 1 line
1.7.6
H. Peter Anvin
reviewerRe: [PATCH v3 6/8] x86-64: Move vread_tsc and vread_hpet into the vDSO
Quoted 4 lines · Andy Lutomirski
The vsyscall page now consists entirely of trap instructions.
Cc John Stultz <[email protected]>Signed-off-by Andy Lutomirski <[email protected]>
This patch causes a build failure on x86-64 allnoconfig:
/home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c: In function ‘vread_hpet’:/home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c:62:
error: implicit declaration of function ‘fix_to_virt’ /home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c:62:
error: ‘VSYSCALL_HPET’ undeclared (first use in this function) /home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c:62:
error: (Each undeclared identifier is reported only once /home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c:62:
error: for each function it appears in.)
make[2]: *** [arch/x86/vdso/vclock_gettime.o] Error 1 make[1]: *** [arch/x86/vdso/vclock_gettime.o] Error 2
make: *** [sub-make] Error 2
Quoted 337 lines
--- arch/x86/include/asm/clocksource.h | 6 +++- arch/x86/include/asm/tsc.h | 4 --- arch/x86/include/asm/vgtod.h | 2 +- arch/x86/include/asm/vsyscall.h | 4 --- arch/x86/kernel/Makefile | 7 +---- arch/x86/kernel/alternative.c | 8 ----- arch/x86/kernel/hpet.c | 9 +----- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 3 -- arch/x86/kernel/vread_tsc_64.c | 36 ------------------------- arch/x86/kernel/vsyscall_64.c | 2 +- arch/x86/vdso/vclock_gettime.c | 52 +++++++++++++++++++++++++++++++---- 12 files changed, 56 insertions(+), 79 deletions(-) delete mode 100644 arch/x86/kernel/vread_tsc_64.carch/x86/include/asm/clocksource.h +5
-1@@ -7,8 +7,12 @@#define __ARCH_HAS_CLOCKSOURCE_DATA +#define VCLOCK_NONE 0/* No vDSO clock available. */ +#define VCLOCK_TSC 1/* vDSO should use vread_tsc. */ +#define VCLOCK_HPET 2/* vDSO should use vread_hpet. */ +struct arch_clocksource_data { -cycle_t (*vread)(void ); +int vclock_mode; };#endif /* CONFIG_X86_64 */ arch/x86/include/asm/tsc.h +0
-4@@ -51,10 +51,6 @@ extern int unsynchronized_tsc(void);extern int check_tsc_unstable (void );extern unsigned long native_calibrate_tsc (void ); -#ifdef CONFIG_X86_64 -extern cycles_t vread_tsc (void ); -#endif -/* * Boot-time check whether the TSCs are synchronized across * all CPUs/cores: arch/x86/include/asm/vgtod.h +1
-1@@ -13,7 +13,7 @@ struct vsyscall_gtod_data {struct timezone sys_tz;struct {/* extract of a clocksource struct */ -cycle_t (*vread)(void ); +int vclock_mode;cycle_t cycle_last;cycle_t mask;u32 mult;arch/x86/include/asm/vsyscall.h +0
-4@@ -16,10 +16,6 @@ enum vsyscall_num {#ifdef __KERNEL__#include <linux/seqlock.h> -/* Definitions for CONFIG_GENERIC_TIME definitions */ -#define __vsyscall_fn \ - __attribute__ ((unused, __section__(".vsyscall_fn"))) notrace -#define VGETCPU_RDTSCP 1#define VGETCPU_LSL 2arch/x86/kernel/Makefile +1
-6@@ -24,17 +24,12 @@ endif nostackp := $(call cc-option, -fno-stack-protector) CFLAGS_vsyscall_64.o := $(PROFILING) -g0 $(nostackp) CFLAGS_hpet.o := $(nostackp) -CFLAGS_vread_tsc_64.o := $(nostackp) CFLAGS_paravirt.o := $(nostackp) GCOV_PROFILE_vsyscall_64.o := n GCOV_PROFILE_hpet.o := n GCOV_PROFILE_tsc.o := n -GCOV_PROFILE_vread_tsc_64.o := n GCOV_PROFILE_paravirt.o := n -# vread_tsc_64 is hot and should be fully optimized: -CFLAGS_REMOVE_vread_tsc_64.o = -pg -fno-optimize-sibling-calls - obj-y := process_$(BITS).o signal.o entry_$(BITS).o obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o ldt.o dumpstack.o@@ -43,7 +38,7 @@ obj-$(CONFIG_IRQ_WORK) += irq_work.o obj-y += probe_roms.o obj-$(CONFIG_X86_32) += sys_i386_32.o i386_ksyms_32.o obj-$(CONFIG_X86_64) += sys_x86_64.o x8664_ksyms_64.o -obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o vread_tsc_64.o +obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o obj-$(CONFIG_X86_64) += vsyscall_emu_64.o obj-y += bootflag.o e820.o obj-y += pci-dma.o quirks.o topology.o kdebugfs.oarch/x86/kernel/alternative.c +0
-8@@ -14,7 +14,6 @@#include <asm/pgtable.h> #include <asm/mce.h> #include <asm/nmi.h> -#include <asm/vsyscall.h> #include <asm/cacheflush.h> #include <asm/tlbflush.h> #include <asm/io.h> @@ -250,7 +249,6 @@ static void __init_or_module add_nops(void *insns, unsigned int len)extern struct alt_instr __alt_instructions[], __alt_instructions_end[];extern s32 __smp_locks[], __smp_locks_end[]; -extern char __vsyscall_0;void *text_poke_early (void *addr,const void *opcode,size_t len);/* Replace instructions with better alternatives for this CPU type. @@ -294,12 +292,6 @@ void __init_or_module apply_alternatives(struct alt_instr *start,add_nops (insnbuf + a->replacementlen, a->instrlen - a->replacementlen); -#ifdef CONFIG_X86_64 -/* vsyscall code is not mapped yet. resolve it manually. */ -if (instr >= (u8 *)VSYSCALL_START && instr < (u8 *)VSYSCALL_END ) { - instr =__va (instr - (u8 *)VSYSCALL_START + (u8 *)__pa_symbol (&__vsyscall_0)); - } -#endif text_poke_early (instr, insnbuf, a->instrlen); } }arch/x86/kernel/hpet.c +1
-8@@ -738,13 +738,6 @@ static cycle_t read_hpet(struct clocksource *cs)return (cycle_t )hpet_readl (HPET_COUNTER ); } -#ifdef CONFIG_X86_64 -static cycle_t __vsyscall_fn vread_hpet(void) -{ -return readl ((const void __iomem *)fix_to_virt (VSYSCALL_HPET ) +0xf0 ); -} -#endif -static struct clocksource clocksource_hpet = { .name ="hpet" , .rating =250 ,@@ -753,7 +746,7 @@ static struct clocksource clocksource_hpet = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .resume = hpet_resume_counter, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_hpet }, + .archdata = { .vclock_mode = VCLOCK_HPET },#endif };arch/x86/kernel/tsc.c +1
-1@@ -777,7 +777,7 @@ static struct clocksource clocksource_tsc = { .flags = CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_MUST_VERIFY, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_tsc }, + .archdata = { .vclock_mode = VCLOCK_TSC },#endif };arch/x86/kernel/vmlinux.lds.S +0
-3@@ -169,9 +169,6 @@ SECTIONS .vsyscall : AT(VLOAD(.vsyscall)) { *(.vsyscall_0) - . = ALIGN(L1_CACHE_BYTES); - *(.vsyscall_fn) - . = 1024; *(.vsyscall_1)arch/x86/kernel/vread_tsc_64.c deleted +0
-36@@ -1,36 +0,0 @@ -/* This code runs in userspace. */ - -#define DISABLE_BRANCH_PROFILING -#include <asm/vgtod.h> - -notrace cycle_t __vsyscall_fn vread_tsc(void) -{ -cycle_t ret; -u64 last; - -/* -* Empirically, a fence (of type that depends on the CPU) -* before rdtsc is enough to ensure that rdtsc is ordered -* with respect to loads. The various CPU manuals are unclear -* as to whether rdtsc can be reordered with later loads, -* but no one has ever seen it happen. -*/ -rdtsc_barrier (); - ret = (cycle_t )vget_cycles (); - - last =VVAR (vsyscall_gtod_data).clock.cycle_last; - -if (likely (ret >= last)) -return ret; - -/* -* GCC likes to generate cmov here, but this branch is extremely -* predictable (it's just a funciton of time and the likely is -* very likely) and there's a data dependence, so force GCC -* to generate a branch instead. I don't barrier() because -* we don't actually need a barrier, and if this function -* ever gets inlined it will generate worse code. -*/ - asmvolatile ("" ); -return last; -}arch/x86/kernel/vsyscall_64.c +1
-1@@ -74,7 +74,7 @@ void update_vsyscall(struct timespec *wall_time, struct timespec *wtm,write_seqlock_irqsave (&vsyscall_gtod_data.lock, flags);/* copy vsyscall data */ - vsyscall_gtod_data.clock.vread = clock->archdata.vread; + vsyscall_gtod_data.clock.vclock_mode = clock->archdata.vclock_mode; vsyscall_gtod_data.clock.cycle_last = clock->cycle_last; vsyscall_gtod_data.clock.mask = clock->mask; vsyscall_gtod_data.clock.mult = mult;arch/x86/vdso/vclock_gettime.c +46
-6@@ -25,6 +25,43 @@#define gtod (&VVAR(vsyscall_gtod_data)) +notrace static cycle_t vread_tsc(void) +{ +cycle_t ret; +u64 last; + +/* +* Empirically, a fence (of type that depends on the CPU) +* before rdtsc is enough to ensure that rdtsc is ordered +* with respect to loads. The various CPU manuals are unclear +* as to whether rdtsc can be reordered with later loads, +* but no one has ever seen it happen. +*/ +rdtsc_barrier (); + ret = (cycle_t )vget_cycles (); + + last =VVAR (vsyscall_gtod_data).clock.cycle_last; + +if (likely (ret >= last)) +return ret; + +/* +* GCC likes to generate cmov here, but this branch is extremely +* predictable (it's just a funciton of time and the likely is +* very likely) and there's a data dependence, so force GCC +* to generate a branch instead. I don't barrier() because +* we don't actually need a barrier, and if this function +* ever gets inlined it will generate worse code. +*/ + asmvolatile ("" ); +return last; +} + +static notrace cycle_t vread_hpet(void) +{ +return readl ((const void __iomem *)fix_to_virt (VSYSCALL_HPET ) +0xf0 ); +} + notrace static long vdso_fallback_gettime(long clock, struct timespec *ts) {long ret;@@ -36,9 +73,12 @@ notrace static long vdso_fallback_gettime(long clock, struct timespec *ts) notrace static inline long vgetns(void) {long v; -cycles_t (*vread)(void); - vread = gtod->clock.vread; - v = (vread () - gtod->clock.cycle_last) & gtod->clock.mask; +cycles_t cycles; +if (gtod->clock.vclock_mode ==VCLOCK_TSC ) + cycles =vread_tsc (); +else + cycles =vread_hpet (); + v = (cycles - gtod->clock.cycle_last) & gtod->clock.mask;return (v * gtod->clock.mult) >> gtod->clock.shift; }@@ -118,11 +158,11 @@ notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts) {switch (clock) {case CLOCK_REALTIME : -if (likely (gtod->clock.vread)) +if (likely (gtod->clock.vclock_mode !=VCLOCK_NONE ))return do_realtime (ts);break ;case CLOCK_MONOTONIC : -if (likely (gtod->clock.vread)) +if (likely (gtod->clock.vclock_mode !=VCLOCK_NONE ))return do_monotonic (ts);break ;case CLOCK_REALTIME_COARSE :@@ -139,7 +179,7 @@ int clock_gettime(clockid_t, struct timespec *) notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz) {long ret; -if (likely (gtod->clock.vread)) { +if (likely (gtod->clock.vclock_mode !=VCLOCK_NONE )) {if (likely (tv !=NULL )) { BUILD_BUG_ON(offsetof(struct timeval, tv_usec) != offsetof(struct timespec, tv_nsec) ||
Re: [PATCH v3] x86-64: Move vread_tsc and vread_hpet into the vDSO
The vsyscall page now consists entirely of trap instructions.
Cc John Stultz <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
---
Quoted 8 lines · H. Peter Anvin
Quoted 4 lines · Andy Lutomirski
The vsyscall page now consists entirely of trap instructions.
Cc John Stultz <[email protected]>Signed-off-by Andy Lutomirski <[email protected]>This patch causes a build failure on x86-64 allnoconfig:
Quoted 14 lines
/home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c: In function 'vread_hpet':/home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c:62:
error: implicit declaration of function 'fix_to_virt'/home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c:62:
error: 'VSYSCALL_HPET' undeclared (first use in this function)/home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c:62:
error: (Each undeclared identifier is reported only once/home/hpa/kernel/linux-2.6-tip.vdso/arch/x86/vdso/vclock_gettime.c:62:
error: for each function it appears in.)
make[2]: *** [arch/x86/vdso/vclock_gettime.o] Error 1 make[1]: *** [arch/x86/vdso/vclock_gettime.o] Error 2
make: *** [sub-make] Error 2
arch/x86/vdso/vclock_gettime.c needs #include <asm/fixmap.h>.
arch/x86/include/asm/clocksource.h | 6 +++- arch/x86/include/asm/tsc.h | 4 --- arch/x86/include/asm/vgtod.h | 2 +- arch/x86/include/asm/vsyscall.h | 4 --- arch/x86/kernel/Makefile | 7 +---- arch/x86/kernel/alternative.c | 8 ----- arch/x86/kernel/hpet.c | 9 +----- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 3 -- arch/x86/kernel/vread_tsc_64.c | 36 ------------------------ arch/x86/kernel/vsyscall_64.c | 2 +- arch/x86/vdso/vclock_gettime.c | 53 +++++++++++++++++++++++++++++++---- 12 files changed, 57 insertions(+), 79 deletions(-) delete mode 100644 arch/x86/kernel/vread_tsc_64.c
arch/x86/include/asm/clocksource.h +5 -1
@@ -7,8 +7,12 @@#define __ARCH_HAS_CLOCKSOURCE_DATA +#define VCLOCK_NONE 0/* No vDSO clock available. */ +#define VCLOCK_TSC 1/* vDSO should use vread_tsc. */ +#define VCLOCK_HPET 2/* vDSO should use vread_hpet. */ +struct arch_clocksource_data { -cycle_t (*vread)(void ); +int vclock_mode; };#endif /* CONFIG_X86_64 */
arch/x86/include/asm/tsc.h +0 -4
@@ -51,10 +51,6 @@ extern int unsynchronized_tsc(void);extern int check_tsc_unstable (void );extern unsigned long native_calibrate_tsc (void ); -#ifdef CONFIG_X86_64 -extern cycles_t vread_tsc (void ); -#endif -/* * Boot-time check whether the TSCs are synchronized across * all CPUs/cores:
arch/x86/include/asm/vgtod.h +1 -1
@@ -13,7 +13,7 @@ struct vsyscall_gtod_data {struct timezone sys_tz;struct {/* extract of a clocksource struct */ -cycle_t (*vread)(void ); +int vclock_mode;cycle_t cycle_last;cycle_t mask;u32 mult;
arch/x86/include/asm/vsyscall.h +0 -4
@@ -16,10 +16,6 @@ enum vsyscall_num {#ifdef __KERNEL__#include <linux/seqlock.h> -/* Definitions for CONFIG_GENERIC_TIME definitions */ -#define __vsyscall_fn \ - __attribute__ ((unused, __section__(".vsyscall_fn"))) notrace -#define VGETCPU_RDTSCP 1#define VGETCPU_LSL 2
arch/x86/kernel/Makefile +1 -6
@@ -24,17 +24,12 @@ endif nostackp := $(call cc-option, -fno-stack-protector) CFLAGS_vsyscall_64.o := $(PROFILING) -g0 $(nostackp) CFLAGS_hpet.o := $(nostackp) -CFLAGS_vread_tsc_64.o := $(nostackp) CFLAGS_paravirt.o := $(nostackp) GCOV_PROFILE_vsyscall_64.o := n GCOV_PROFILE_hpet.o := n GCOV_PROFILE_tsc.o := n -GCOV_PROFILE_vread_tsc_64.o := n GCOV_PROFILE_paravirt.o := n -# vread_tsc_64 is hot and should be fully optimized: -CFLAGS_REMOVE_vread_tsc_64.o = -pg -fno-optimize-sibling-calls - obj-y := process_$(BITS).o signal.o entry_$(BITS).o obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o ldt.o dumpstack.o
@@ -43,7 +38,7 @@ obj-$(CONFIG_IRQ_WORK) += irq_work.o obj-y += probe_roms.o obj-$(CONFIG_X86_32) += sys_i386_32.o i386_ksyms_32.o obj-$(CONFIG_X86_64) += sys_x86_64.o x8664_ksyms_64.o -obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o vread_tsc_64.o +obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o obj-$(CONFIG_X86_64) += vsyscall_emu_64.o obj-y += bootflag.o e820.o obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
arch/x86/kernel/alternative.c +0 -8
@@ -14,7 +14,6 @@#include <asm/pgtable.h> #include <asm/mce.h> #include <asm/nmi.h> -#include <asm/vsyscall.h> #include <asm/cacheflush.h> #include <asm/tlbflush.h> #include <asm/io.h>
@@ -250,7 +249,6 @@ static void __init_or_module add_nops(void *insns, unsigned int len)extern struct alt_instr __alt_instructions[], __alt_instructions_end[];extern s32 __smp_locks[], __smp_locks_end[]; -extern char __vsyscall_0;void *text_poke_early (void *addr,const void *opcode,size_t len);/* Replace instructions with better alternatives for this CPU type.
@@ -294,12 +292,6 @@ void __init_or_module apply_alternatives(struct alt_instr *start,add_nops (insnbuf + a->replacementlen, a->instrlen - a->replacementlen); -#ifdef CONFIG_X86_64 -/* vsyscall code is not mapped yet. resolve it manually. */ -if (instr >= (u8 *)VSYSCALL_START && instr < (u8 *)VSYSCALL_END ) { - instr =__va (instr - (u8 *)VSYSCALL_START + (u8 *)__pa_symbol (&__vsyscall_0)); - } -#endif text_poke_early (instr, insnbuf, a->instrlen); } }
arch/x86/kernel/hpet.c +1 -8
@@ -738,13 +738,6 @@ static cycle_t read_hpet(struct clocksource *cs)return (cycle_t )hpet_readl (HPET_COUNTER ); } -#ifdef CONFIG_X86_64 -static cycle_t __vsyscall_fn vread_hpet(void) -{ -return readl ((const void __iomem *)fix_to_virt (VSYSCALL_HPET ) +0xf0 ); -} -#endif -static struct clocksource clocksource_hpet = { .name ="hpet" , .rating =250 ,
@@ -753,7 +746,7 @@ static struct clocksource clocksource_hpet = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .resume = hpet_resume_counter, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_hpet }, + .archdata = { .vclock_mode = VCLOCK_HPET },#endif };
arch/x86/kernel/tsc.c +1 -1
@@ -777,7 +777,7 @@ static struct clocksource clocksource_tsc = { .flags = CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_MUST_VERIFY, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_tsc }, + .archdata = { .vclock_mode = VCLOCK_TSC },#endif };
arch/x86/kernel/vmlinux.lds.S +0 -3
@@ -169,9 +169,6 @@ SECTIONS .vsyscall : AT(VLOAD(.vsyscall)) { *(.vsyscall_0) - . = ALIGN(L1_CACHE_BYTES); - *(.vsyscall_fn) - . = 1024; *(.vsyscall_1)
arch/x86/kernel/vread_tsc_64.c deleted +0 -36
@@ -1,36 +0,0 @@ -/* This code runs in userspace. */ - -#define DISABLE_BRANCH_PROFILING -#include <asm/vgtod.h> - -notrace cycle_t __vsyscall_fn vread_tsc(void) -{ -cycle_t ret; -u64 last; - -/* -* Empirically, a fence (of type that depends on the CPU) -* before rdtsc is enough to ensure that rdtsc is ordered -* with respect to loads. The various CPU manuals are unclear -* as to whether rdtsc can be reordered with later loads, -* but no one has ever seen it happen. -*/ -rdtsc_barrier (); - ret = (cycle_t )vget_cycles (); - - last =VVAR (vsyscall_gtod_data).clock.cycle_last; - -if (likely (ret >= last)) -return ret; - -/* -* GCC likes to generate cmov here, but this branch is extremely -* predictable (it's just a funciton of time and the likely is -* very likely) and there's a data dependence, so force GCC -* to generate a branch instead. I don't barrier() because -* we don't actually need a barrier, and if this function -* ever gets inlined it will generate worse code. -*/ - asmvolatile ("" ); -return last; -}
arch/x86/kernel/vsyscall_64.c +1 -1
@@ -74,7 +74,7 @@ void update_vsyscall(struct timespec *wall_time, struct timespec *wtm,write_seqlock_irqsave (&vsyscall_gtod_data.lock, flags);/* copy vsyscall data */ - vsyscall_gtod_data.clock.vread = clock->archdata.vread; + vsyscall_gtod_data.clock.vclock_mode = clock->archdata.vclock_mode; vsyscall_gtod_data.clock.cycle_last = clock->cycle_last; vsyscall_gtod_data.clock.mask = clock->mask; vsyscall_gtod_data.clock.mult = mult;
arch/x86/vdso/vclock_gettime.c +47 -6
86 lines not on this page · raw message
Signature · 1 line
1.7.6
[tip:x86/vdso] x86-64: Move vread_tsc and vread_hpet into the vDSO
Commit-ID: 98d0ac38ca7b1b7a552c9a2359174ff84decb600 Gitweb: http://git.kernel.org/tip/98d0ac38ca7b1b7a552c9a2359174ff84decb600 Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 14 Jul 2011 06:47:22 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Thu, 14 Jul 2011 17:57:05 -0700
x86-64: Move vread_tsc and vread_hpet into the vDSO
The vsyscall page now consists entirely of trap instructions.
Cc John Stultz <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
Link http://lkml.kernel.org/r/637648f303f2ef93af93bae25186e9a1bea093f5.1310639973.git.luto@mit.edu open here
Signed-off-by H. Peter Anvin <[email protected]>
--- arch/x86/include/asm/clocksource.h | 6 +++- arch/x86/include/asm/tsc.h | 4 --- arch/x86/include/asm/vgtod.h | 2 +- arch/x86/include/asm/vsyscall.h | 4 --- arch/x86/kernel/Makefile | 7 +---- arch/x86/kernel/alternative.c | 8 ----- arch/x86/kernel/hpet.c | 9 +----- arch/x86/kernel/tsc.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 3 -- arch/x86/kernel/vread_tsc_64.c | 36 ------------------------ arch/x86/kernel/vsyscall_64.c | 2 +- arch/x86/vdso/vclock_gettime.c | 53 +++++++++++++++++++++++++++++++---- 12 files changed, 57 insertions(+), 79 deletions(-)
arch/x86/include/asm/clocksource.h +5 -1
@@ -7,8 +7,12 @@#define __ARCH_HAS_CLOCKSOURCE_DATA +#define VCLOCK_NONE 0/* No vDSO clock available. */ +#define VCLOCK_TSC 1/* vDSO should use vread_tsc. */ +#define VCLOCK_HPET 2/* vDSO should use vread_hpet. */ +struct arch_clocksource_data { -cycle_t (*vread)(void ); +int vclock_mode; };#endif /* CONFIG_X86_64 */
arch/x86/include/asm/tsc.h +0 -4
@@ -51,10 +51,6 @@ extern int unsynchronized_tsc(void);extern int check_tsc_unstable (void );extern unsigned long native_calibrate_tsc (void ); -#ifdef CONFIG_X86_64 -extern cycles_t vread_tsc (void ); -#endif -/* * Boot-time check whether the TSCs are synchronized across * all CPUs/cores:
arch/x86/include/asm/vgtod.h +1 -1
@@ -13,7 +13,7 @@ struct vsyscall_gtod_data {struct timezone sys_tz;struct {/* extract of a clocksource struct */ -cycle_t (*vread)(void ); +int vclock_mode;cycle_t cycle_last;cycle_t mask;u32 mult;
arch/x86/include/asm/vsyscall.h +0 -4
@@ -16,10 +16,6 @@ enum vsyscall_num {#ifdef __KERNEL__#include <linux/seqlock.h> -/* Definitions for CONFIG_GENERIC_TIME definitions */ -#define __vsyscall_fn \ - __attribute__ ((unused, __section__(".vsyscall_fn"))) notrace -#define VGETCPU_RDTSCP 1#define VGETCPU_LSL 2
arch/x86/kernel/Makefile +1 -6
@@ -24,17 +24,12 @@ endif nostackp := $(call cc-option, -fno-stack-protector) CFLAGS_vsyscall_64.o := $(PROFILING) -g0 $(nostackp) CFLAGS_hpet.o := $(nostackp) -CFLAGS_vread_tsc_64.o := $(nostackp) CFLAGS_paravirt.o := $(nostackp) GCOV_PROFILE_vsyscall_64.o := n GCOV_PROFILE_hpet.o := n GCOV_PROFILE_tsc.o := n -GCOV_PROFILE_vread_tsc_64.o := n GCOV_PROFILE_paravirt.o := n -# vread_tsc_64 is hot and should be fully optimized: -CFLAGS_REMOVE_vread_tsc_64.o = -pg -fno-optimize-sibling-calls - obj-y := process_$(BITS).o signal.o entry_$(BITS).o obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o ldt.o dumpstack.o
@@ -43,7 +38,7 @@ obj-$(CONFIG_IRQ_WORK) += irq_work.o obj-y += probe_roms.o obj-$(CONFIG_X86_32) += sys_i386_32.o i386_ksyms_32.o obj-$(CONFIG_X86_64) += sys_x86_64.o x8664_ksyms_64.o -obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o vread_tsc_64.o +obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o obj-$(CONFIG_X86_64) += vsyscall_emu_64.o obj-y += bootflag.o e820.o obj-y += pci-dma.o quirks.o topology.o kdebugfs.o
arch/x86/kernel/alternative.c +0 -8
@@ -14,7 +14,6 @@#include <asm/pgtable.h> #include <asm/mce.h> #include <asm/nmi.h> -#include <asm/vsyscall.h> #include <asm/cacheflush.h> #include <asm/tlbflush.h> #include <asm/io.h>
@@ -250,7 +249,6 @@ static void __init_or_module add_nops(void *insns, unsigned int len)extern struct alt_instr __alt_instructions[], __alt_instructions_end[];extern s32 __smp_locks[], __smp_locks_end[]; -extern char __vsyscall_0;void *text_poke_early (void *addr,const void *opcode,size_t len);/* Replace instructions with better alternatives for this CPU type.
@@ -294,12 +292,6 @@ void __init_or_module apply_alternatives(struct alt_instr *start,add_nops (insnbuf + a->replacementlen, a->instrlen - a->replacementlen); -#ifdef CONFIG_X86_64 -/* vsyscall code is not mapped yet. resolve it manually. */ -if (instr >= (u8 *)VSYSCALL_START && instr < (u8 *)VSYSCALL_END ) { - instr =__va (instr - (u8 *)VSYSCALL_START + (u8 *)__pa_symbol (&__vsyscall_0)); - } -#endif text_poke_early (instr, insnbuf, a->instrlen); } }
arch/x86/kernel/hpet.c +1 -8
@@ -738,13 +738,6 @@ static cycle_t read_hpet(struct clocksource *cs)return (cycle_t )hpet_readl (HPET_COUNTER ); } -#ifdef CONFIG_X86_64 -static cycle_t __vsyscall_fn vread_hpet(void) -{ -return readl ((const void __iomem *)fix_to_virt (VSYSCALL_HPET ) +0xf0 ); -} -#endif -static struct clocksource clocksource_hpet = { .name ="hpet" , .rating =250 ,
@@ -753,7 +746,7 @@ static struct clocksource clocksource_hpet = { .flags = CLOCK_SOURCE_IS_CONTINUOUS, .resume = hpet_resume_counter, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_hpet }, + .archdata = { .vclock_mode = VCLOCK_HPET },#endif };
arch/x86/kernel/tsc.c +1 -1
@@ -777,7 +777,7 @@ static struct clocksource clocksource_tsc = { .flags = CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_MUST_VERIFY, #ifdef CONFIG_X86_64 - .archdata = { .vread = vread_tsc }, + .archdata = { .vclock_mode = VCLOCK_TSC },#endif };
arch/x86/kernel/vmlinux.lds.S +0 -3
@@ -169,9 +169,6 @@ SECTIONS .vsyscall : AT(VLOAD(.vsyscall)) { *(.vsyscall_0) - . = ALIGN(L1_CACHE_BYTES); - *(.vsyscall_fn) - . = 1024; *(.vsyscall_1)
arch/x86/kernel/vread_tsc_64.c deleted +0 -36
@@ -1,36 +0,0 @@ -/* This code runs in userspace. */ - -#define DISABLE_BRANCH_PROFILING -#include <asm/vgtod.h> - -notrace cycle_t __vsyscall_fn vread_tsc(void) -{ -cycle_t ret; -u64 last; - -/* -* Empirically, a fence (of type that depends on the CPU) -* before rdtsc is enough to ensure that rdtsc is ordered -* with respect to loads. The various CPU manuals are unclear -* as to whether rdtsc can be reordered with later loads, -* but no one has ever seen it happen. -*/ -rdtsc_barrier (); - ret = (cycle_t )vget_cycles (); - - last =VVAR (vsyscall_gtod_data).clock.cycle_last; - -if (likely (ret >= last)) -return ret; - -/* -* GCC likes to generate cmov here, but this branch is extremely -* predictable (it's just a funciton of time and the likely is -* very likely) and there's a data dependence, so force GCC -* to generate a branch instead. I don't barrier() because -* we don't actually need a barrier, and if this function -* ever gets inlined it will generate worse code. -*/ - asmvolatile ("" ); -return last; -}
arch/x86/kernel/vsyscall_64.c +1 -1
@@ -74,7 +74,7 @@ void update_vsyscall(struct timespec *wall_time, struct timespec *wtm,write_seqlock_irqsave (&vsyscall_gtod_data.lock, flags);/* copy vsyscall data */ - vsyscall_gtod_data.clock.vread = clock->archdata.vread; + vsyscall_gtod_data.clock.vclock_mode = clock->archdata.vclock_mode; vsyscall_gtod_data.clock.cycle_last = clock->cycle_last; vsyscall_gtod_data.clock.mask = clock->mask; vsyscall_gtod_data.clock.mult = mult;
arch/x86/vdso/vclock_gettime.c +47 -6
86 lines not on this page · raw message
PATCH v3 7/8 ia64: Replace clocksource.fsys_mmio with generic arch data
In reply to Andy LutomirskiNow that clocksource.archdata is available, use it for ia64-specific code.
Cc Clemens Ladisch <[email protected]>
Cc Tony Luck <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc John Stultz <[email protected]>
Cc Thomas Gleixner <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/ia64/include/asm/clocksource.h | 12 ++++++++++++ arch/ia64/kernel/cyclone.c | 2 +- arch/ia64/kernel/time.c | 2 +- arch/ia64/sn/kernel/sn2/timer.c | 2 +- drivers/char/hpet.c | 2 +- include/linux/clocksource.h | 7 ------- 6 files changed, 16 insertions(+), 11 deletions(-) create mode 100644 arch/ia64/include/asm/clocksource.h
arch/ia64/include/asm/clocksource.h new file +12 -0
@@ -0,0 +1,12 @@ +/* IA64-specific clocksource additions */ + +#ifndef _ASM_IA64_CLOCKSOURCE_H +#define _ASM_IA64_CLOCKSOURCE_H + +#define __ARCH_HAS_CLOCKSOURCE_DATA + +struct arch_clocksource_data { +void *fsys_mmio;/* used by fsyscall asm code */ +}; + +#endif /* _ASM_IA64_CLOCKSOURCE_H */
arch/ia64/kernel/cyclone.c +1 -1
@@ -115,7 +115,7 @@ int __init init_cyclone_clock(void) }/* initialize last tick */ cyclone_mc = cyclone_timer; - clocksource_cyclone.fsys_mmio = cyclone_timer; + clocksource_cyclone.archdata.fsys_mmio = cyclone_timer;clocksource_register_hz (&clocksource_cyclone,CYCLONE_TIMER_FREQ );return 0 ;
arch/ia64/kernel/time.c +1 -1
@@ -468,7 +468,7 @@ void update_vsyscall(struct timespec *wall, struct timespec *wtm, fsyscall_gtod_data.clk_mask = c->mask; fsyscall_gtod_data.clk_mult = mult; fsyscall_gtod_data.clk_shift = c->shift; - fsyscall_gtod_data.clk_fsys_mmio = c->fsys_mmio; + fsyscall_gtod_data.clk_fsys_mmio = c->archdata.fsys_mmio; fsyscall_gtod_data.clk_cycle_last = c->cycle_last;/* copy kernel time structures */
arch/ia64/sn/kernel/sn2/timer.c +1 -1
@@ -54,7 +54,7 @@ ia64_sn_udelay (unsigned long usecs)void __init sn_timer_init (void ) { - clocksource_sn2.fsys_mmio =RTC_COUNTER_ADDR ; + clocksource_sn2.archdata.fsys_mmio =RTC_COUNTER_ADDR ;clocksource_register_hz (&clocksource_sn2, sn_rtc_cycles_per_second); ia64_udelay = &ia64_sn_udelay;
drivers/char/hpet.c +1 -1
@@ -952,7 +952,7 @@ int hpet_alloc(struct hpet_data *hdp)#ifdef CONFIG_IA64 if (!hpet_clocksource) { hpet_mctr = (void __iomem *)&hpetp->hp_hpet->hpet_mc; -CLKSRC_FSYS_MMIO_SET (clocksource_hpet.fsys_mmio, hpet_mctr); + clocksource_hpet.archdata.fsys_mmio = hpet_mctr;clocksource_register_hz (&clocksource_hpet, hpetp->hp_tick_freq); hpetp->hp_clocksource = &clocksource_hpet; hpet_clocksource = &clocksource_hpet;
include/linux/clocksource.h +0 -7
@@ -171,13 +171,6 @@ struct clocksource {u32 shift;u64 max_idle_ns; -#ifdef CONFIG_IA64 -void *fsys_mmio;/* used by fsyscall asm code */ -#define CLKSRC_FSYS_MMIO_SET (mmio, addr) ((mmio) = (addr)) -#else -#define CLKSRC_FSYS_MMIO_SET (mmio, addr) do { } while (0) -#endif -#ifdef __ARCH_HAS_CLOCKSOURCE_DATAstruct arch_clocksource_data archdata;#endif
Signature · 1 line
1.7.6
[tip:x86/vdso] ia64: Replace clocksource.fsys_mmio with generic arch data
Commit-ID: 574c44fa8fa6262ffd5939789ef51a6e98ed62d7 Gitweb: http://git.kernel.org/tip/574c44fa8fa6262ffd5939789ef51a6e98ed62d7 Author: Andy Lutomirski <[email protected]> AuthorDate: Wed, 13 Jul 2011 09:24:15 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Thu, 14 Jul 2011 17:57:09 -0700
ia64: Replace clocksource.fsys_mmio with generic arch data
Now that clocksource.archdata is available, use it for ia64-specific code.
Cc Clemens Ladisch <[email protected]>
Cc Tony Luck <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc John Stultz <[email protected]>
Cc Thomas Gleixner <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
Link http://lkml.kernel.org/r/d31de0ee0842a0e322fb6441571c2b0adb323fa2.1310563276.git.luto@mit.edu open here
Signed-off-by H. Peter Anvin <[email protected]>
--- arch/ia64/include/asm/clocksource.h | 12 ++++++++++++ arch/ia64/kernel/cyclone.c | 2 +- arch/ia64/kernel/time.c | 2 +- arch/ia64/sn/kernel/sn2/timer.c | 2 +- drivers/char/hpet.c | 2 +- include/linux/clocksource.h | 7 ------- 6 files changed, 16 insertions(+), 11 deletions(-)
arch/ia64/include/asm/clocksource.h new file +12 -0
@@ -0,0 +1,12 @@ +/* IA64-specific clocksource additions */ + +#ifndef _ASM_IA64_CLOCKSOURCE_H +#define _ASM_IA64_CLOCKSOURCE_H + +#define __ARCH_HAS_CLOCKSOURCE_DATA + +struct arch_clocksource_data { +void *fsys_mmio;/* used by fsyscall asm code */ +}; + +#endif /* _ASM_IA64_CLOCKSOURCE_H */
arch/ia64/kernel/cyclone.c +1 -1
@@ -115,7 +115,7 @@ int __init init_cyclone_clock(void) }/* initialize last tick */ cyclone_mc = cyclone_timer; - clocksource_cyclone.fsys_mmio = cyclone_timer; + clocksource_cyclone.archdata.fsys_mmio = cyclone_timer;clocksource_register_hz (&clocksource_cyclone,CYCLONE_TIMER_FREQ );return 0 ;
arch/ia64/kernel/time.c +1 -1
@@ -468,7 +468,7 @@ void update_vsyscall(struct timespec *wall, struct timespec *wtm, fsyscall_gtod_data.clk_mask = c->mask; fsyscall_gtod_data.clk_mult = mult; fsyscall_gtod_data.clk_shift = c->shift; - fsyscall_gtod_data.clk_fsys_mmio = c->fsys_mmio; + fsyscall_gtod_data.clk_fsys_mmio = c->archdata.fsys_mmio; fsyscall_gtod_data.clk_cycle_last = c->cycle_last;/* copy kernel time structures */
arch/ia64/sn/kernel/sn2/timer.c +1 -1
@@ -54,7 +54,7 @@ ia64_sn_udelay (unsigned long usecs)void __init sn_timer_init (void ) { - clocksource_sn2.fsys_mmio =RTC_COUNTER_ADDR ; + clocksource_sn2.archdata.fsys_mmio =RTC_COUNTER_ADDR ;clocksource_register_hz (&clocksource_sn2, sn_rtc_cycles_per_second); ia64_udelay = &ia64_sn_udelay;
drivers/char/hpet.c +1 -1
@@ -931,7 +931,7 @@ int hpet_alloc(struct hpet_data *hdp)#ifdef CONFIG_IA64 if (!hpet_clocksource) { hpet_mctr = (void __iomem *)&hpetp->hp_hpet->hpet_mc; -CLKSRC_FSYS_MMIO_SET (clocksource_hpet.fsys_mmio, hpet_mctr); + clocksource_hpet.archdata.fsys_mmio = hpet_mctr;clocksource_register_hz (&clocksource_hpet, hpetp->hp_tick_freq); hpetp->hp_clocksource = &clocksource_hpet; hpet_clocksource = &clocksource_hpet;
include/linux/clocksource.h +0 -7
@@ -171,13 +171,6 @@ struct clocksource {u32 shift;u64 max_idle_ns; -#ifdef CONFIG_IA64 -void *fsys_mmio;/* used by fsyscall asm code */ -#define CLKSRC_FSYS_MMIO_SET (mmio, addr) ((mmio) = (addr)) -#else -#define CLKSRC_FSYS_MMIO_SET (mmio, addr) do { } while (0) -#endif -#ifdef __ARCH_HAS_CLOCKSOURCE_DATAstruct arch_clocksource_data archdata;#endif
PATCH v3 8/8 Document the vDSO and add a reference parser
In reply to Andy LutomirskiIt turns out that parsing the vDSO is nontrivial if you don't already have an ELF dynamic loader around. So document it in Documentation/ABI and add a reference CC0-licenced parser.
This code is dedicated to Go issue 1933:
http://code.google.com/p/go/issues/detail?id=1933
Signed-off-by Andy Lutomirski <[email protected]>
--- Documentation/ABI/stable/vdso | 27 ++++ Documentation/vDSO/parse_vdso.c | 256 +++++++++++++++++++++++++++++++++++++++ Documentation/vDSO/vdso_test.c | 112 +++++++++++++++++ 3 files changed, 395 insertions(+), 0 deletions(-) create mode 100644 Documentation/ABI/stable/vdso create mode 100644 Documentation/vDSO/parse_vdso.c create mode 100644 Documentation/vDSO/vdso_test.c
Documentation/ABI/stable/vdso new file +27 -0
@@ -0,0 +1,27 @@ +On some architectures, when the kernel loads any userspace program it +maps an ELF DSO into that program's address space. This DSO is called +the vDSO and it often contains useful and highly-optimized alternatives +to real syscalls. + +These functions are called just like ordinary C function according to +your platform's ABI. Call them from a sensible context. (For example, +if you set CS on x86 to something strange, the vDSO functions are +within their rights to crash.) In addition, if you pass a bad +pointer to a vDSO function, you might get SIGSEGV instead of -EFAULT. + +To find the DSO, parse the auxiliary vector passed to the program's +entry point. The AT_SYSINFO_EHDR entry will point to the vDSO. + +The vDSO uses symbol versioning; whenever you request a symbol from the +vDSO, specify the version you are expecting. + +Programs that dynamically link to glibc will use the vDSO automatically. +Otherwise, you can use the reference parser in Documentation/vDSO/parse_vdso.c. + +Unless otherwise noted, the set of symbols with any given version and the +ABI of those symbols is considered stable. It may vary across architectures, +though. + +(As of this writing, this ABI documentation as been confirmed for x86_64. + The maintainers of the other vDSO-using architectures should confirm + that it is correct for their architecture.)
Documentation/vDSO/parse_vdso.c new file +256 -0
256 lines not on this page · raw message
Documentation/vDSO/vdso_test.c new file +112 -0
112 lines not on this page · raw message
Signature · 1 line
1.7.6
[tip:x86/vdso] Document the vDSO and add a reference parser
Commit-ID: 98eedc3a9dbf90cecb91093d2a7fa083942b7d13 Gitweb: http://git.kernel.org/tip/98eedc3a9dbf90cecb91093d2a7fa083942b7d13 Author: Andy Lutomirski <[email protected]> AuthorDate: Wed, 13 Jul 2011 09:24:16 -0400 Committer: H. Peter Anvin <[email protected]> CommitDate: Thu, 14 Jul 2011 17:57:09 -0700
Document the vDSO and add a reference parser
It turns out that parsing the vDSO is nontrivial if you don't already have an ELF dynamic loader around. So document it in Documentation/ABI and add a reference CC0-licenced parser.
This code is dedicated to Go issue 1933:
http://code.google.com/p/go/issues/detail?id=1933
Signed-off-by Andy Lutomirski <[email protected]>
Link http://lkml.kernel.org/r/a315a9514cd71bcf29436cc31e35aada21a5ff21.1310563276.git.luto@mit.edu open here
Signed-off-by H. Peter Anvin <[email protected]>
--- Documentation/ABI/stable/vdso | 27 ++++ Documentation/vDSO/parse_vdso.c | 256 +++++++++++++++++++++++++++++++++++++++ Documentation/vDSO/vdso_test.c | 111 +++++++++++++++++ 3 files changed, 394 insertions(+), 0 deletions(-)
Documentation/ABI/stable/vdso new file +27 -0
@@ -0,0 +1,27 @@ +On some architectures, when the kernel loads any userspace program it +maps an ELF DSO into that program's address space. This DSO is called +the vDSO and it often contains useful and highly-optimized alternatives +to real syscalls. + +These functions are called just like ordinary C function according to +your platform's ABI. Call them from a sensible context. (For example, +if you set CS on x86 to something strange, the vDSO functions are +within their rights to crash.) In addition, if you pass a bad +pointer to a vDSO function, you might get SIGSEGV instead of -EFAULT. + +To find the DSO, parse the auxiliary vector passed to the program's +entry point. The AT_SYSINFO_EHDR entry will point to the vDSO. + +The vDSO uses symbol versioning; whenever you request a symbol from the +vDSO, specify the version you are expecting. + +Programs that dynamically link to glibc will use the vDSO automatically. +Otherwise, you can use the reference parser in Documentation/vDSO/parse_vdso.c. + +Unless otherwise noted, the set of symbols with any given version and the +ABI of those symbols is considered stable. It may vary across architectures, +though. + +(As of this writing, this ABI documentation as been confirmed for x86_64. + The maintainers of the other vDSO-using architectures should confirm + that it is correct for their architecture.)
Documentation/vDSO/parse_vdso.c new file +256 -0
256 lines not on this page · raw message
Documentation/vDSO/vdso_test.c new file +111 -0
111 lines not on this page · raw message