ARM: VDSO
Subsystems (2)
v10 cover ARM: VDSO
Please consider this a call for review and test results. I believe this code is stable and ready for inclusion, but patches 3-6 lack explicit acks/reviews so far.
Russell, I plan to add at least the sigpage changes (patches 1 and 2) to your patch system shortly after this posting, as they have Reviewed-by tags and are useful independent of the VDSO.
Provide fast userspace implementations of gettimeofday and clock_gettime on systems that implement the generic timers extension defined in ARMv7. This follows the example of arm64 in conception but significantly differs in some aspects of the implementation (C vs assembly, mainly).
Clocks supported:
- CLOCK_REALTIME
- CLOCK_MONOTONIC
- CLOCK_REALTIME_COARSE
- CLOCK_MONOTONIC_COARSE
This also provides clock_getres (as arm64 does).
getcpu support is planned but not included at this time.
For applications to transparently benefit from this change, ARM-specific support code needs to be added to glibc. I hope to have that code added to glibc for the 2.21 release.
The VDSO symbols are available for lookup via dlsym even with an unpatched glibc.
Note that while the high-precision realtime and monotonic clock support depends on the generic timers extension, support for clock_getres and coarse clocks is independent of the timer implementation and is provided unconditionally. High-resolution clock support requires changes to the arch timer code, posted here:
http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/288452.html
The VDSO will function correctly without those changes, but gettimeofday and clock_gettime with CLOCK_REALTIME/CLOCK_MONOTONIC will not be accelerated.
Tested on OMAP5 and i.MX6, verifying that results obtained with the vdso are consistent with those obtained from the kernel. On OMAP5 I observe a 3- to 4-fold speedup for gettimeofday / CLOCK_REALTIME, with even better (if less interesting) speedups for the coarse clock ids and clock_getres.
I've been testing and benchmarking this with some custom test code which I have hosted here:
https://github.com/nlynch-mentor/vdsotest
Unpatched FSF GDB may complain "warning: Could not load shared library symbols for linux-vdso.so.1." This is not an ARM-specific issue. Current Fedora and Debian patch their GDB packages to prevent this warning.
Changes since v9:
- Update to 3.17-rc5.
- Fixed minor checkpatch warnings (line length etc).
- Removed useless VDSO_LBASE constant.
- Added 'notrace' attributes to VDSO-internal functions.
- Removed unused debug code from vdsomunge host program.
Changes since v8:
- Update to 3.17-rc1.
- Split out arch timer changes into separate series.
- Ensure that VDSO will not attempt to read the counter if access is not enabled; this check can be removed after the arch timer changes are merged. See update_vsyscall and vdso_can_use_arch_timer in patch #5.
Changes since v7:
- Update to next-20140801.
- In arch_setup_additional_pages, fix call to get_unmapped_area - use bytes (not pages) for length argument.
- As x86 does, separate data and text into two VMAs, [vvar] and [vdso] respectively. These have different permissions; [vdso] will allow debuggers to set breakpoints, but [vvar] is read-only and cannot be modified even via ptrace.
- Use _install_special_mapping for signal page, vvar, and vdso mappings.
- Add -DDISABLE_BRANCH_PROFILING.
- Add --no-undefined -Bsymbolic to link options to cause linker to error out on unresolved references, making checkundef script unnecessary.
- Specify max-page-size, common-page-size in linker options so the true alignment is reflected in program header; otherwise gdb gets confused.
- Fix incremental build vs. generated auxvec.h.
- Use appropriate unwind directives in __get_datapage.
- Added vdso_install target and help text. Install build-id symlinks as x86 does.
- Adjust update_vsyscall for changes in struct timekeeper.
Changes since v6:
- Update to 3.16-rc1.
- Remove -lgcc from link step - need to support GCC installations without libgcc.
- Force -O2 compilation to prevent GCC from emitting calls to libgcc math routines.
- Use custom post-processing to clear the EF_ARM_ABI_FLOAT_SOFT flag if set in the ELF header to produce a shared object which is architecturally allowed to be used by both soft- and hard-float code.
- Consolidate common arch timer code instead of duplicating it.
- Prevent the VDSO from attempting CP15 access on memory-only architected timer implementations by renaming the clocksource.
Changes since v5:
- Update to 3.15-rc1.
- Place vdso at a randomized offset above the stack along with the sigpage.
- Properly export asm/auxvec.h.
- Split patch into series for ease of review.
Changes since v4:
- Map data page at the beginning of the VMA to prevent orphan sections at the end of output invalidating the calculated offset.
- Move checkundef into cmd_vdsold to avoid spurious rebuilds.
- Change vdso_init message to pr_debug.
- Add -fno-stack-protector to cflags.
Changes since v3:
- Update to 3.14-rc6.
- Record vdso base in mm context before installing mapping (for the sake of perf_mmap_event).
- Use a more seqcount-like API for critical sections. Using seqcount API directly, however, would leak kernel pointers to userspace when lockdep is enabled.
- Trap instead of looping forever in division-by-zero stubs.
Changes since v2:
- Update to 3.14-rc4.
- Make vDSO configurable, depending on AEABI and MMU.
- Defer shifting of nanosecond component of timespec: fixes observed 1ns inconsistencies for CLOCK_REALTIME, CLOCK_MONOTONIC (see 45a7905fc48f for arm64 equivalent).
- Force reload of seq_count when spinning: without a memory clobber after the load of vdata->seq_count, GCC can generate code like this:
2f8: e59c9020 ldr r9, [ip, #32]
2fc: e3190001 tst r9, #1
300: 1a000033 bne 3d4 <do_realtime+0x104>
304: f57ff05b dmb ish
308: e59c3034 ldr r3, [ip, #52] ; 0x34
...
3d4: eafffffe b 3d4 <do_realtime+0x104>- Build vdso.so with -lgcc: calls to __lshrdi3, __divsi3 sometimes emitted (especially with -Os). Override certain libgcc functions to prevent undefined symbols.
- Do not clear PG_reserved on vdso pages.
- Remove unnecessary get_page calls.
- Simplify ELF signature check during init.
- Use volatile for asm syscall fallbacks.
- Check whether vdso_pagelist is initialized in arm_install_vdso.
- Record clocksource mask in data page.
- Reduce code duplication in do_realtime, do_monotonic.
- Reduce calculations performed in critical sections.
- Simplify coarse clock handling.
- Move datapage load to its own assembly routine.
- Tune vdso_data layout and tweak field names.
- Check vdso shared object for undefined symbols during build.
Changes since v1:
- update to 3.14-rc1
- ensure cache coherency for data page
- Document the kernel-to-userspace protocol for vdso data page updates, and note that the timekeeping core prevents concurrent updates.
- update wall-to-monotonic fields unconditionally
- move vdso_start, vdso_end declarations to vdso.h
- correctly build and run when CONFIG_ARM_ARCH_TIMER=n
- rearrange linker script to avoid overlapping sections when CONFIG_DEBUGINFO=n
- remove use_syscall checks from coarse clock paths
- crib BUG_INSTR (0xe7f001f2) from asm/bug.h for text fill
Nathan Lynch (6):
ARM: use _install_special_mapping for sigpage
ARM: place sigpage at a random offset above stack
ARM: miscellaneous vdso infrastructure, preparation
ARM: add vdso user-space code
ARM: vdso initialization, mapping, and synchronization
ARM: add CONFIG_VDSO Kconfig and Makefile bits
arch/arm/Makefile | 8 + arch/arm/include/asm/Kbuild | 1 - arch/arm/include/asm/auxvec.h | 1 + arch/arm/include/asm/elf.h | 9 + arch/arm/include/asm/mmu.h | 3 + arch/arm/include/asm/vdso.h | 32 ++++ arch/arm/include/asm/vdso_datapage.h | 60 +++++++ arch/arm/include/uapi/asm/Kbuild | 1 + arch/arm/include/uapi/asm/auxvec.h | 7 + arch/arm/kernel/Makefile | 1 + arch/arm/kernel/asm-offsets.c | 5 + arch/arm/kernel/process.c | 72 +++++++- arch/arm/kernel/vdso.c | 211 ++++++++++++++++++++++ arch/arm/mm/Kconfig | 15 ++ arch/arm/vdso/.gitignore | 1 + arch/arm/vdso/Makefile | 74 ++++++++ arch/arm/vdso/datapage.S | 15 ++ arch/arm/vdso/vdso.S | 35 ++++ arch/arm/vdso/vdso.lds.S | 88 ++++++++++ arch/arm/vdso/vdsomunge.c | 201 +++++++++++++++++++++ arch/arm/vdso/vgettimeofday.c | 328 +++++++++++++++++++++++++++++++++++ 21 files changed, 1158 insertions(+), 10 deletions(-) create mode 100644 arch/arm/include/asm/auxvec.h create mode 100644 arch/arm/include/asm/vdso.h create mode 100644 arch/arm/include/asm/vdso_datapage.h create mode 100644 arch/arm/include/uapi/asm/auxvec.h create mode 100644 arch/arm/kernel/vdso.c create mode 100644 arch/arm/vdso/.gitignore create mode 100644 arch/arm/vdso/Makefile create mode 100644 arch/arm/vdso/datapage.S create mode 100644 arch/arm/vdso/vdso.S create mode 100644 arch/arm/vdso/vdso.lds.S create mode 100644 arch/arm/vdso/vdsomunge.c create mode 100644 arch/arm/vdso/vgettimeofday.c
Signature · 1 line
1.9.3
PATCH v10 1/6 ARM: use _install_special_mapping for sigpage
_install_special_mapping allows the VMA to be identifed in /proc/pid/maps without the use of arch_vma_name, providing a slight net reduction in object size:
text data bss dec hex filename 2996 96 144 3236 ca4 arch/arm/kernel/process.o (before) 2956 104 144 3204 c84 arch/arm/kernel/process.o (after)
--- arch/arm/kernel/process.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
arch/arm/kernel/process.c +16 -8
@@ -472,19 +472,23 @@ int in_gate_area_no_mm(unsigned long addr)const char *arch_vma_name (struct vm_area_struct *vma) { -return is_gate_vma (vma) ?"[vectors]" : - (vma->vm_mm && vma->vm_start == vma->vm_mm->context.sigpage) ? -"[sigpage]" :NULL ; +return is_gate_vma (vma) ?"[vectors]" :NULL ; }static struct page *signal_page;extern struct page *get_signal_page (void ); +static const struct vm_special_mapping sigpage_mapping = { + .name ="[sigpage]" , + .pages = &signal_page, +}; +int arch_setup_additional_pages (struct linux_binprm *bprm,int uses_interp) {struct mm_struct *mm = current->mm; +struct vm_area_struct *vma;unsigned long addr; -int ret; +int ret =0 ;if (!signal_page) signal_page =get_signal_page ();
@@ -498,12 +502,16 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) goto up_fail; } - ret =install_special_mapping (mm, addr,PAGE_SIZE , + vma =_install_special_mapping (mm, addr,PAGE_SIZE ,VM_READ |VM_EXEC |VM_MAYREAD |VM_MAYWRITE |VM_MAYEXEC , - &signal_page); + &sigpage_mapping); + +if (IS_ERR (vma)) { + ret =PTR_ERR (vma); + goto up_fail; + } -if (ret ==0 ) - mm->context.sigpage = addr; + mm->context.sigpage = addr; up_fail:up_write (&mm->mmap_sem);
Signature · 1 line
1.9.3
PATCH v10 2/6 ARM: place sigpage at a random offset above stack
In reply to Nathan LynchThe sigpage is currently placed alongside shared libraries etc in the address space. Similar to what x86_64 does for its VDSO, place the sigpage at a randomized offset above the stack so that learning the base address of the sigpage doesn't help expose where shared libraries are loaded in the address space (and vice versa).
--- arch/arm/kernel/process.c | 37 ++++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-)
arch/arm/kernel/process.c +36 -1
@@ -475,6 +475,39 @@ const char *arch_vma_name(struct vm_area_struct *vma)return is_gate_vma (vma) ?"[vectors]" :NULL ; } +/* If possible, provide a placement hint at a random offset from the +* stack for the signal page. +*/ +static unsigned long sigpage_addr (const struct mm_struct *mm, +unsigned int npages) +{ +unsigned long offset; +unsigned long first; +unsigned long last; +unsigned long addr; +unsigned int slots; + + first =PAGE_ALIGN (mm->start_stack); + + last =TASK_SIZE - (npages <<PAGE_SHIFT ); + +/* No room after stack? */ +if (first > last) +return 0 ; + +/* Just enough room? */ +if (first == last) +return first; + + slots = ((last - first) >>PAGE_SHIFT ) +1 ; + + offset =get_random_int () % slots; + + addr = first + (offset <<PAGE_SHIFT ); + +return addr; +} +static struct page *signal_page;extern struct page *get_signal_page (void );
@@ -488,6 +521,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)struct mm_struct *mm = current->mm;struct vm_area_struct *vma;unsigned long addr; +unsigned long hint;int ret =0 ;if (!signal_page)
@@ -496,7 +530,8 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)return -ENOMEM ;down_write (&mm->mmap_sem); - addr =get_unmapped_area (NULL ,0 ,PAGE_SIZE ,0 ,0 ); + hint =sigpage_addr (mm,1 ); + addr =get_unmapped_area (NULL , hint,PAGE_SIZE ,0 ,0 );if (IS_ERR_VALUE (addr)) { ret = addr; goto up_fail;
Signature · 1 line
1.9.3
PATCH v10 3/6 ARM: miscellaneous vdso infrastructure, preparation
In reply to Nathan LynchDefine the layout of the data structure shared between kernel and userspace.
Track the vdso address in the mm_context; needed for communicating AT_SYSINFO_EHDR to the ELF loader.
Add declarations for arm_install_vdso; implementation is in a following patch.
Define AT_SYSINFO_EHDR, and, if CONFIG_VDSO=y, report the vdso shared object address via the ELF auxiliary vector.
Note - this adds the AT_SYSINFO_EHDR in a new user-visible header asm/auxvec.h; this is consistent with other architectures.
--- arch/arm/include/asm/Kbuild | 1 - arch/arm/include/asm/auxvec.h | 1 + arch/arm/include/asm/elf.h | 9 ++++++ arch/arm/include/asm/mmu.h | 3 ++ arch/arm/include/asm/vdso.h | 32 +++++++++++++++++++ arch/arm/include/asm/vdso_datapage.h | 60 ++++++++++++++++++++++++++++++++++++ arch/arm/include/uapi/asm/Kbuild | 1 + arch/arm/include/uapi/asm/auxvec.h | 7 +++++ 8 files changed, 113 insertions(+), 1 deletion(-) create mode 100644 arch/arm/include/asm/auxvec.h create mode 100644 arch/arm/include/asm/vdso.h create mode 100644 arch/arm/include/asm/vdso_datapage.h create mode 100644 arch/arm/include/uapi/asm/auxvec.h
arch/arm/include/asm/Kbuild +0 -1
@@ -1,6 +1,5 @@ -generic-y += auxvec.h generic-y += bitsperlong.h generic-y += cputime.h generic-y += current.h
arch/arm/include/asm/auxvec.h new file +1 -0
@@ -0,0 +1 @@ +#include <uapi/asm/auxvec.h>
arch/arm/include/asm/elf.h +9 -0
@@ -1,7 +1,9 @@#ifndef __ASMARM_ELF_H#define __ASMARM_ELF_H +#include <asm/auxvec.h> #include <asm/hwcap.h> +#include <asm/vdso_datapage.h> /* * ELF register definitions..
@@ -130,6 +132,13 @@ extern unsigned long arch_randomize_brk(struct mm_struct *mm);#define arch_randomize_brk arch_randomize_brk#ifdef CONFIG_MMU +#ifdef CONFIG_VDSO +#define ARCH_DLINFO \ +do { \ + NEW_AUX_ENT(AT_SYSINFO_EHDR, \ + (elf_addr_t)current->mm->context.vdso); \ +} while (0) +#endif #define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1struct linux_binprm ;int arch_setup_additional_pages (struct linux_binprm *,int );
arch/arm/include/asm/mmu.h +3 -0
@@ -11,6 +11,9 @@ typedef struct {#endif unsigned int vmalloc_seq;unsigned long sigpage; +#ifdef CONFIG_VDSO +unsigned long vdso; +#endif } mm_context_t;#ifdef CONFIG_CPU_HAS_ASID
arch/arm/include/asm/vdso.h new file +32 -0
@@ -0,0 +1,32 @@ +#ifndef __ASM_VDSO_H +#define __ASM_VDSO_H + +#ifdef __KERNEL__ + +#ifndef __ASSEMBLY__ + +struct mm_struct ; + +#ifdef CONFIG_VDSO + +void arm_install_vdso (struct mm_struct *mm,unsigned long addr); + +extern char vdso_start, vdso_end; + +extern unsigned int vdso_total_pages; + +#else /* CONFIG_VDSO */ + +static inline void arm_install_vdso (struct mm_struct *mm,unsigned long addr) +{ +} + +#define vdso_total_pages 0 + +#endif /* CONFIG_VDSO */ + +#endif /* __ASSEMBLY__ */ + +#endif /* __KERNEL__ */ + +#endif /* __ASM_VDSO_H */
arch/arm/include/asm/vdso_datapage.h new file +60 -0
60 lines not on this page · raw message
arch/arm/include/uapi/asm/Kbuild +1 -0
@@ -1,6 +1,7 @@ # UAPI Header export list include include/uapi/asm-generic/Kbuild.asm +header-y += auxvec.h header-y += byteorder.h header-y += fcntl.h header-y += hwcap.h
arch/arm/include/uapi/asm/auxvec.h new file +7 -0
@@ -0,0 +1,7 @@ +#ifndef __ASM_AUXVEC_H +#define __ASM_AUXVEC_H + +/* vDSO location */ +#define AT_SYSINFO_EHDR 33 + +#endif
Signature · 1 line
1.9.3
PATCH v10 4/6 ARM: add vdso user-space code
In reply to Nathan LynchPlace vdso-related user-space code in arch/arm/kernel/vdso/.
It is almost completely written in C with some assembly helpers to load the data page address, sample the counter, and fall back to system calls when necessary.
If CONFIG_ARM_ARCH_TIMER is not enabled, the vdso cannot service high-resolution clocks and falls back to syscalls. Low-resolution clocks e.g. CLOCK_REALTIME_COARSE can be serviced regardless.
Of particular note is that a post-processing step ("vdsomunge") is necessary to produce a shared object which is architecturally allowed to be used by both soft- and hard-float EABI programs.
The 2012 edition of the ARM ABI defines Tag_ABI_VFP_args = 3 "Code is compatible with both the base and VFP variants; the user did not permit non-variadic functions to pass FP parameters/results." Unfortunately current toolchains do not support this tag, which is ideally what we would use.
The best available option is to ensure that both EF_ARM_ABI_FLOAT_SOFT and EF_ARM_ABI_FLOAT_HARD are unset in the ELF header's e_flags, indicating that the shared object is "old" and should be accepted for backward compatibility's sake. While binutils < 2.24 appear to produce a vdso.so with both flags clear, 2.24 always sets EF_ARM_ABI_FLOAT_SOFT, with no way to inhibit this behavior. So we have to fix things up with a custom post-processing step.
In fact, the VDSO code in glibc does much less validation (including checking these flags) than the code for handling conventional file-backed shared libraries, so this is a bit moot unless glibc's VDSO code becomes more strict.
--- arch/arm/kernel/asm-offsets.c | 5 + arch/arm/vdso/.gitignore | 1 + arch/arm/vdso/Makefile | 74 ++++++++++ arch/arm/vdso/datapage.S | 15 ++ arch/arm/vdso/vdso.S | 35 +++++ arch/arm/vdso/vdso.lds.S | 88 ++++++++++++ arch/arm/vdso/vdsomunge.c | 201 ++++++++++++++++++++++++++ arch/arm/vdso/vgettimeofday.c | 328 ++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 747 insertions(+) create mode 100644 arch/arm/vdso/.gitignore create mode 100644 arch/arm/vdso/Makefile create mode 100644 arch/arm/vdso/datapage.S create mode 100644 arch/arm/vdso/vdso.S create mode 100644 arch/arm/vdso/vdso.lds.S create mode 100644 arch/arm/vdso/vdsomunge.c create mode 100644 arch/arm/vdso/vgettimeofday.c
arch/arm/kernel/asm-offsets.c +5 -0
@@ -24,6 +24,7 @@#include <asm/memory.h> #include <asm/procinfo.h> #include <asm/suspend.h> +#include <asm/vdso_datapage.h> #include <asm/hardware/cache-l2x0.h> #include <linux/kbuild.h>
@@ -200,5 +201,9 @@ int main(void)#endif DEFINE(KVM_VTTBR, offsetof(struct kvm, arch.vttbr));#endif +BLANK (); +#ifdef CONFIG_VDSO +DEFINE (VDSO_DATA_SIZE ,sizeof (union vdso_data_store )); +#endif return 0 ; }
arch/arm/vdso/.gitignore new file +1 -0
@@ -0,0 +1 @@ +vdso.lds
arch/arm/vdso/Makefile new file +74 -0
74 lines not on this page · raw message
arch/arm/vdso/datapage.S new file +15 -0
@@ -0,0 +1,15 @@ +#include <linux/linkage.h> +#include <asm/asm-offsets.h> + + .align 2 +.L_vdso_data_ptr: + .long _start - . - VDSO_DATA_SIZE + +ENTRY(__get_datapage) + .fnstart + adr r0, .L_vdso_data_ptr + ldr r1, [r0] + add r0, r0, r1 + bx lr + .fnend +ENDPROC(__get_datapage)
arch/arm/vdso/vdso.S new file +35 -0
@@ -0,0 +1,35 @@ +/* + * Adapted from arm64 version. + * + * Copyright (C) 2012 ARM Limited + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + * + * Author: Will Deacon <[email protected]> + */ + +#include <linux/init.h> +#include <linux/linkage.h> +#include <linux/const.h> +#include <asm/page.h> + + __PAGE_ALIGNED_DATA + + .globl vdso_start, vdso_end + .balign PAGE_SIZE +vdso_start: + .incbin "arch/arm/vdso/vdso.so" + .balign PAGE_SIZE +vdso_end: + + .previous
arch/arm/vdso/vdso.lds.S new file +88 -0
88 lines not on this page · raw message
arch/arm/vdso/vdsomunge.c new file +201 -0
201 lines not on this page · raw message
arch/arm/vdso/vgettimeofday.c new file +328 -0
328 lines not on this page · raw message
Signature · 1 line
1.9.3
PATCH v10 4/6 ARM: add vdso user-space code
arch/arm/vdso/vgettimeofday.c line 126
Quoted 9 lines · Nathan Lynch <nathan_lynch <at> mentor.com> writes:
arch/arm/vdso/vgettimeofday.c new file +9
-0+#ifdef CONFIG_ARM_ARCH_TIMER + +static notrace u64 get_ns(struct vdso_data *vdata) +{ +u64 cycle_delta; +u64 cycle_now; +u64 nsec; + + cycle_now =arch_counter_get_cntvct ();
Hello Nathan,
Regarding ARM:vDSO, I have enabled your patch set on Cortex-A7/ARMv7, the vDSO is up and running using generic arch timer using physical counter but not with VCT.
I anticipate, the issue could be due to VCT cycles.
let me explain the problem first,
The current validation system at the following state,
commit 65b5732d241b8b39e07653794eefffd0d8028cbb(branch: linux-3.19.y)
clocksource: arch_timer: Allow the device tree to specify
uninitialized timer registers
Some 32-bit (ARMv7) systems are architected like this:
* The firmware doesn't know and doesn't care about hypervisor mode and we don't want to add the complexity of hypervisor there.
* The firmware isn't involved in SMP bringup or resume.
* The ARCH timer come up with an uninitialized offset (CNTVOFF) between the virtual and physical counters. Each core gets a different random offset.
* The device boots in "Secure SVC" mode .
On systems like the above, it doesn't make sense to use the virtual counter. There's nobody managing the offset and each time a core goes down and comes back up it will get reinitialized to some other random value.
Based on the above, physical timer like this really only makes sense for ARMv7 systems.
To support vDSO on the above configuration,
Will arch_counter_get_cntvct() work?
OR
Should we use arch_counter_get_cntpct()?
Perhaps, either we need to revise the vDSO frame work to choose VCT or PCT dynamically Otherwise CNTVOFF reset to be zero but CNTVOFF could not be accessed in SVC mode (PL1).
BR,
Venkappa
PATCH v10 4/6 ARM: add vdso user-space code
Quoted 16 lines · Venkappa Mala
arch/arm/vdso/vgettimeofday.c line 126
Quoted 9 lines · Nathan Lynch <nathan_lynch <at> mentor.com> writes:
arch/arm/vdso/vgettimeofday.c new file +9
-0+#ifdef CONFIG_ARM_ARCH_TIMER + +static notrace u64 get_ns(struct vdso_data *vdata) +{ +u64 cycle_delta; +u64 cycle_now; +u64 nsec; + + cycle_now =arch_counter_get_cntvct ();Regarding ARM:vDSO, I have enabled your patch set on Cortex-A7/ARMv7, the vDSO is up and running using generic arch timer using physical counter but not with VCT.
I anticipate, the issue could be due to VCT cycles.
[...]
Quoted 3 lines
Perhaps, either we need to revise the vDSO frame work to choose VCT or PCT dynamically Otherwise CNTVOFF reset to be zero but CNTVOFF could not be accessed in SVC mode (PL1).
I assume on your platform, the DT also contains "arm,cpu-registers-not-fw-configured". In such case, the VDSO optimisation for gettimeofday should just be disabled (as it would be if the generic timers are not present). Really, just fix the firmware.
Signature · 1 line
Catalin
PATCH v10 4/6 ARM: add vdso user-space code
Quoted 4 lines · Catalin Marinas <catalin.marinas <at> arm.com> writes:
Quoted 2 lines · Venkappa Mala
Nathan Lynch <
nathan_lynch<at> mentor.com> writes:
Quoted 11 lines
Quoted 11 lines
arch/arm/vdso/vgettimeofday.c line 126
Quoted 9 lines
arch/arm/vdso/vgettimeofday.c new file +9
-0+#ifdef CONFIG_ARM_ARCH_TIMER + +static notrace u64 get_ns(struct vdso_data *vdata) +{ +u64 cycle_delta; +u64 cycle_now; +u64 nsec; + + cycle_now =arch_counter_get_cntvct ();Regarding ARM:vDSO, I have enabled your patch set on Cortex-
A7/ARMv7,
Quoted 5 lines
Quoted 3 lines
the vDSO is up and running using generic arch timer using physical counter but not with VCT.
I anticipate, the issue could be due to VCT cycles.[...]
Quoted 1 line
Perhaps, either we need to revise the vDSO frame work to choose VCT
or
Quoted 1 line
Quoted 1 line
PCT dynamically Otherwise CNTVOFF reset to be zero but CNTVOFF could
not
Quoted 5 lines
Quoted 1 line
be accessed in SVC mode (PL1).
I assume on your platform, the DT also contains "arm,cpu-registers-not-fw-configured". In such case, the VDSO optimisation for gettimeofday should just be disabled (as it would be
if
Quoted 2 lines
the generic timers are not present). Really, just fix the firmware.
Thanks Catalin for your promote reply.
Yes, DTS contained arm,cpu-registers-not-fw-configured so that it uses generic physical timer instead of virtual timer.
Now, the vDSO gettimeofday optimization is also possible since the generic timer is presented and enabled.
However, vDSO gettimeofday implementation has been using virtual timer (VCT user access) but the current implementation
will not be sufficient to handle when the firmware is not taken care about reset CNTVOFF to zero.
Let?s assume,neither firmware taken care nor booted with HYP mode, then Can we use physical timer (PCT user access needs to be enabled) for vDSO in the safe manner?
PATCH v10 4/6 ARM: add vdso user-space code
Quoted 27 lines · Venkappa Mala
Quoted 17 lines · Catalin Marinas <catalin.marinas <at> arm.com> writes:
Quoted 15 lines · Venkappa Mala
arch/arm/vdso/vgettimeofday.c line 126
Quoted 9 lines · Nathan Lynch <nathan_lynch <at> mentor.com> writes:
arch/arm/vdso/vgettimeofday.c new file +9
-0+#ifdef CONFIG_ARM_ARCH_TIMER + +static notrace u64 get_ns(struct vdso_data *vdata) +{ +u64 cycle_delta; +u64 cycle_now; +u64 nsec; + + cycle_now =arch_counter_get_cntvct ();Regarding ARM:vDSO, I have enabled your patch set on Cortex-A7/ARMv7, the vDSO is up and running using generic arch timer using physical counter but not with VCT.
I anticipate, the issue could be due to VCT cycles.[...]
Quoted 8 lines
Quoted 3 lines
Perhaps, either we need to revise the vDSO frame work to choose VCT or PCT dynamically Otherwise CNTVOFF reset to be zero but CNTVOFF could not be accessed in SVC mode (PL1).
I assume on your platform, the DT also contains "arm,cpu-registers-not-fw-configured". In such case, the VDSO optimisation for gettimeofday should just be disabled (as it would be if the generic timers are not present). Really, just fix the firmware.
Quoted 10 lines
Thanks Catalin for your promote reply. Yes, DTS contained arm,cpu-registers-not-fw-configured so that it uses generic physical timer instead of virtual timer. Now, the vDSO gettimeofday optimization is also possible since the generic timer is presented and enabled. However, vDSO gettimeofday implementation has been using virtual timer (VCT user access) but the current implementation will not be sufficient to handle when the firmware is not taken care about reset CNTVOFF to zero. Let?s assume,neither firmware taken care nor booted with HYP mode, then Can we use physical timer (PCT user access needs to be enabled) for vDSO in the safe manner?
The short answer: no.
The problem is that user space is very "innovative" in making use of features in a different way than what the kernel people intended and it will be claimed "user ABI" afterwards. Take this glibc revert for example, luckily we noticed it early:
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=33ef2f0c763b51e1df7896d7d39d585824558c75
CNTVCT is not intended to be used outside of the VDSO but, apparently, the VDSO->kernel interface is seen as user ABI by some.
CNTPCT cannot always be used in user space (VDSO and not even the kernel), especially when the kernel boots at EL1. Choosing CNTPCT vs CNTVCT in VDSO dynamically (or at boot time) is technically doable but there is a high risk that user space ends up using them directly when it detects a gettimeofday VDSO and such code will break when CNTPCT is no longer accessible.
Signature · 1 line
Catalin
PATCH v10 4/6 ARM: add vdso user-space code
Quoted 5 lines · Catalin Marinas <catalin.marinas <at> arm.com> writes:
Quoted 3 lines · Venkappa Mala
Quoted 2 lines · Catalin Marinas <catalin.marinas <at> arm.com> writes:
Quoted 1 line · Venkappa Mala
Nathan Lynch <
nathan_lynch<at> mentor.com> writes:
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
arch/arm/vdso/vgettimeofday.c line 126
Quoted 9 lines
arch/arm/vdso/vgettimeofday.c new file +9
-0+#ifdef CONFIG_ARM_ARCH_TIMER + +static notrace u64 get_ns(struct vdso_data *vdata) +{ +u64 cycle_delta; +u64 cycle_now; +u64 nsec; + + cycle_now =arch_counter_get_cntvct ();Regarding ARM:vDSO, I have enabled your patch set on Cortex-
A7/ARMv7,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
the vDSO is up and running using generic arch timer using
physical
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
counter but not with VCT.
I anticipate, the issue could be due to VCT cycles.[...]
Quoted 1 line
Perhaps, either we need to revise the vDSO frame work to choose
VCT or
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
PCT dynamically Otherwise CNTVOFF reset to be zero but CNTVOFF
could not
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
be accessed in SVC mode (PL1).
I assume on your platform, the DT also contains "arm,cpu-registers-not-fw-configured". In such case, the VDSO optimisation for gettimeofday should just be disabled (as it would
be if
Quoted 1 line
Quoted 1 line
Quoted 1 line
the generic timers are not present). Really, just fix the
firmware.
Quoted 5 lines
Quoted 5 lines
Thanks Catalin for your promote reply. Yes, DTS contained arm,cpu-registers-not-fw-configured so that it uses generic physical timer instead of virtual timer. Now, the vDSO gettimeofday optimization is also possible since the generic timer is presented
and
Quoted 3 lines
Quoted 3 lines
enabled. However, vDSO gettimeofday implementation has been using virtual timer (VCT user access) but the current implementation will not be sufficient to handle when the firmware is not taken care
about
Quoted 2 lines
Quoted 2 lines
reset CNTVOFF to zero. Let?s assume,neither firmware taken care nor booted with HYP mode, then Can we use physical timer (PCT user
access
Quoted 10 lines
Quoted 1 line
needs to be enabled) for vDSO in the safe manner?
The short answer: no.
The problem is that user space is very "innovative" in making use of features in a different way than what the kernel people intended and it will be claimed "user ABI" afterwards. Take this glibc revert for example, luckily we noticed it early:
p=glibc.git;a=commitdiff;h=33ef2f0c763b51e1df7896d7d39d585824558c75
Quoted 8 lines
CNTVCT is not intended to be used outside of the VDSO but, apparently, the
VDSO->kernelinterface is seen as user ABI by some.CNTPCT cannot always be used in user space (VDSO and not even the kernel), especially when the kernel boots at EL1. Choosing CNTPCT vs CNTVCT in VDSO dynamically (or at boot time) is technically doable but there is a high risk that user space ends up using them directly when
it
Quoted 3 lines
detects a gettimeofday VDSO and such code will break when CNTPCT is no longer accessible.
Ok Catalin, I have understood about your concern.
Quoted 1 line
From ARMv7 TRM,
An implementation of the Generic Timer always includes a virtual counter, that indicates virtual time:
[1] In a processor implementation that does not include the Virtualization Extensions, virtual time is identical to physical time, and the virtual counter contains the same value as the physical counter.
[2] If In a processor implementation that includes the Virtualization Extensions, then it is already taken care in the kernel with the following if boot loader boots with HYP mode,
commit 0af0b189abf73d232af782df2f999235cd2fed7f
ARM: hyp: initialize CNTVOFF to zero
Currently, we have been noticing an issue with [1] with virtual offset when we enable VCT since it is not an initialize to zero. Perhaps, the best way to handle this one in firmware itself rather than bringing too much complexity.
Are you recommending this one?
If that is true, then we are going to force everyone to take care CNTVOFF initialization in the firmware if they want to utilize vDSO on ARMv7 where HYP mode is disabled.
BR,
Venkappa
PATCH v10 4/6 ARM: add vdso user-space code
Quoted 40 lines · Venkappa Mala
Quoted 24 lines · Catalin Marinas <catalin.marinas <at> arm.com> writes:
Quoted 10 lines · Venkappa Mala
Thanks Catalin for your promote reply. Yes, DTS contained arm,cpu-registers-not-fw-configured so that it uses generic physical timer instead of virtual timer. Now, the vDSO gettimeofday optimization is also possible since the generic timer is presented and enabled. However, vDSO gettimeofday implementation has been using virtual timer (VCT user access) but the current implementation will not be sufficient to handle when the firmware is not taken care about reset CNTVOFF to zero. Let?s assume,neither firmware taken care nor booted with HYP mode, then Can we use physical timer (PCT user access needs to be enabled) for vDSO in the safe manner?
The short answer: no.
The problem is that user space is very "innovative" in making use of features in a different way than what the kernel people intended and it will be claimed "user ABI" afterwards. Take this glibc revert for example, luckily we noticed it early:
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=33ef2f0c763b51e1df7896d7d39d585824558c75
CNTVCT is not intended to be used outside of the VDSO but, apparently, the
VDSO->kernelinterface is seen as user ABI by some.Quoted 6 lines
CNTPCT cannot always be used in user space (VDSO and not even the kernel), especially when the kernel boots at EL1. Choosing CNTPCT vs CNTVCT in VDSO dynamically (or at boot time) is technically doable but there is a high risk that user space ends up using them directly when it detects a gettimeofday VDSO and such code will break when CNTPCT is no longer accessible.
Ok Catalin, I have understood about your concern.
From ARMv7 TRM,
An implementation of the Generic Timer always includes a virtual counter, that indicates virtual time:
[1] In a processor implementation that does not include the Virtualization Extensions, virtual time is identical to physical time, and the virtual counter contains the same value as the physical counter.
Quoted 8 lines
[2] If In a processor implementation that includes the Virtualization Extensions, then it is already taken care in the kernel with the following if boot loader boots with HYP mode,
commit0af0b189abf73d232af782df2f999235cd2fed7f
ARM: hyp: initialize CNTVOFF to zeroCurrently, we have been noticing an issue with [1] with virtual offset when we enable VCT since it is not an initialize to zero.
I highly doubt you are having a problem with [1] since you mentioned Cortex-A7 which falls under [2]. If you have a _processor_ implementation with generic timers but no Hyp, as per the ARMv7 ARM the CNTVCT == CNTPCT, otherwise it is a CPU bug.
What you actually have is [2] but without code to initialise the Hyp mode. If your kernel is running non-secure, sane Hyp registers initialisation is mandatory even when the kernel does not boot in Hyp.
Quoted 3 lines
Perhaps, the best way to handle this one in firmware itself rather than bringing too much complexity.
Are you recommending this one?
Yes, definitely. But note that it's not only (secondary) boot but coming out of a lower power mode which normally involves firmware anyway.
Quoted 3 lines
If that is true, then we are going to force everyone to take care CNTVOFF initialization in the firmware if they want to utilize vDSO on ARMv7 where HYP mode is disabled.
Even for the kernel, this should be the case unless "arm,cpu-registers-not-fw-configured" is passed in the DT (this was really only meant as workaround for broken firmware; I think Documentation/arm/Booting needs to be updated with the state of the generic timers on entry).
IIRC there were some discussions about Linux booting in secure mode on ARMv7 with no firmware to initialise the Hyp registers. If that's the case for you, we could restart that discussion and try to initialise CNTVOFF in the kernel running in _secure_ mode (possibly with some CPU notifiers in the generic timer code).
Signature · 1 line
Catalin
PATCH v10 4/6 ARM: add vdso user-space code
Quoted 3 lines · Catalin Marinas <catalin.marinas <at> arm.com> writes:
I highly doubt you are having a problem with [1] since you mentioned Cortex-A7 which falls under [2]. If you have a _processor_ implementation with generic timers but no Hyp, as per the ARMv7 ARM
the
Quoted 5 lines
CNTVCT == CNTPCT, otherwise it is a CPU bug.
What you actually have is [2] but without code to initialise the Hyp mode. If your kernel is running non-secure, sane Hyp registers initialisation is mandatory even when the kernel does not boot in Hyp.
yes, cortexA7 is falling under [2]. i.e. my mistake. Thanks for the correction.
Quoted 5 lines
Even for the kernel, this should be the case unless "arm,cpu-registers-not-fw-configured" is passed in the DT (this was really only meant as workaround for broken firmware; I think
Documentation/arm/Bootingneeds to be updated with the state of the generic timers on entry).
It will be good and it helps a lot if it updates.
Quoted 5 lines
IIRC there were some discussions about Linux booting in secure mode on ARMv7 with no firmware to initialise the Hyp registers. If that's the case for you, we could restart that discussion and try to initialise CNTVOFF in the kernel running in _secure_ mode (possibly with some CPU notifiers in the generic timer code).
My test environment is like CNTVOFF initialization is not taken care in the firmware, HYP mode is disabled and kernel is booted with Secure SVC PL1 mode. As you have mentioned about restarting the thread, perhaps, it will be good to have a discussion in the following thread rather than restarting the thread.
arm: Handle starting up in secure mode
https://lkml.org/lkml/2014/9/16/552
May I know your opinion about this one?
BR,
Venkappa
PATCH v10 4/6 ARM: add vdso user-space code
Hi Venkappa,
Quoted 35 lines · Venkappa Mala
Quoted 3 lines · Catalin Marinas <catalin.marinas <at> arm.com> writes:
I highly doubt you are having a problem with [1] since you mentioned Cortex-A7 which falls under [2]. If you have a _processor_ implementation with generic timers but no Hyp, as per the ARMv7 ARM
the
Quoted 5 lines
CNTVCT == CNTPCT, otherwise it is a CPU bug.
What you actually have is [2] but without code to initialise the Hyp mode. If your kernel is running non-secure, sane Hyp registers initialisation is mandatory even when the kernel does not boot in Hyp.
yes, cortexA7 is falling under [2]. i.e. my mistake. Thanks for the correction.
Quoted 5 lines
Even for the kernel, this should be the case unless "arm,cpu-registers-not-fw-configured" is passed in the DT (this was really only meant as workaround for broken firmware; I think
Documentation/arm/Bootingneeds to be updated with the state of the generic timers on entry).It will be good and it helps a lot if it updates.
Quoted 5 lines
IIRC there were some discussions about Linux booting in secure mode on ARMv7 with no firmware to initialise the Hyp registers. If that's the case for you, we could restart that discussion and try to initialise CNTVOFF in the kernel running in _secure_ mode (possibly with some CPU notifiers in the generic timer code).
My test environment is like CNTVOFF initialization is not taken care in the firmware, HYP mode is disabled and kernel is booted with Secure SVC PL1 mode. As you have mentioned about restarting the thread, perhaps, it will be good to have a discussion in the following thread rather than restarting the thread.
arm: Handle starting up in secure mode
https://lkml.org/lkml/2014/9/16/552
Quoted 1 line
May I know your opinion about this one?
Catalin pointed out that accesses to NSACR may trap. I have not yet found a reason why trying to write to CNTFRQ wouldn't work. Peter pointed out that VBAR can't be used on systems without the security extensions, so something has to be changed there as well.
In the long term I may be able to post an updated version, but I have no idea how long that might take. In the short term feel free to CC me on any related work you do, which I might be able to review, test, or both.
Chris
Signature · 3 lines
Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
PATCH v10 4/6 ARM: add vdso user-space code
Quoted 16 lines · Christopher Covington
Quoted 15 lines · Venkappa Mala
Quoted 5 lines · Catalin Marinas <catalin.marinas <at> arm.com> writes:
IIRC there were some discussions about Linux booting in secure mode on ARMv7 with no firmware to initialise the Hyp registers. If that's the case for you, we could restart that discussion and try to initialise CNTVOFF in the kernel running in _secure_ mode (possibly with some CPU notifiers in the generic timer code).
My test environment is like CNTVOFF initialization is not taken care in the firmware, HYP mode is disabled and kernel is booted with Secure SVC PL1 mode. As you have mentioned about restarting the thread, perhaps, it will be good to have a discussion in the following thread rather than restarting the thread.
arm: Handle starting up in secure mode
https://lkml.org/lkml/2014/9/16/552
Quoted 4 lines
Quoted 1 line
May I know your opinion about this one?
Catalin pointed out that accesses to NSACR may trap. I have not yet found a reason why trying to write to CNTFRQ wouldn't work.
Writing CNTFRQ from non-secure mode would trap as well. For whatever reason, the architecture doesn't allow you to easily detect whether the code runs in secure or non-secure mode (unless it is EL2/Hyp which is always non-secure). The detection mechanism would have to involve trapping of an undefined instruction but it doesn't need to be done early during boot.
Otherwise, I think in the arch timer code we could do something like:
if (!is_hyp_mode_available() && !arch_timer_user_virtual &&
arch_timer_ppi[PHYS_SECURE_PPI]) {
/*
* Assume the kernel is running in secure mode. Program
* the CNVOFF and CNTFRQ registers.
*/
...
}Signature · 1 line
Catalin
PATCH v10 4/6 ARM: add vdso user-space code
Hi Catalin,
On 02/24/2015 05:17 AM, Catalin Marinas wrote:
[snip]
Quoted 1 line
Writing CNTFRQ from non-secure mode would trap as well.
Are you saying it could A) cause an exception targeting the current EL/mode but not others, or B) cause an exception targeting the current or another EL/mode?
[snip]
Quoted 2 lines
The detection mechanism would have to involve trapping of an undefined instruction but it doesn't need to be done early during boot.
I put detection during early boot because I wanted to potentially change ELs/modes during early boot.
Thanks,
Chris
Signature · 3 lines
Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
PATCH v10 4/6 ARM: add vdso user-space code
Quoted 5 lines · Christopher Covington
Quoted 1 line · Catalin Marinas
Writing CNTFRQ from non-secure mode would trap as well.
Are you saying it could A) cause an exception targeting the current EL/mode but not others, or B) cause an exception targeting the current or another EL/mode?
IIRC, it only traps at the current mode (if non-secure EL1 or EL2).
Quoted 5 lines
Quoted 2 lines
The detection mechanism would have to involve trapping of an undefined instruction but it doesn't need to be done early during boot.
I put detection during early boot because I wanted to potentially change ELs/modes during early boot.
Yes, but it is simpler to install an undef hook later in the boot process.
Signature · 1 line
Catalin
PATCH v10 4/6 ARM: add vdso user-space code
In reply to Venkappa MalaQuoted 14 lines · Venkappa Mala
arch/arm/vdso/vgettimeofday.c line 126
Quoted 9 lines · Nathan Lynch <nathan_lynch <at> mentor.com> writes:
arch/arm/vdso/vgettimeofday.c new file +9
-0+#ifdef CONFIG_ARM_ARCH_TIMER + +static notrace u64 get_ns(struct vdso_data *vdata) +{ +u64 cycle_delta; +u64 cycle_now; +u64 nsec; + + cycle_now =arch_counter_get_cntvct ();Hello Nathan,
Quoted 3 lines
Regarding ARM:vDSO, I have enabled your patch set on Cortex-A7/ARMv7, the vDSO is up and running using generic arch timer using physical counter but not with VCT.
Hi,
Sorry for not seeing this earlier but it doesn't look like my address was in the recipients list, and I just happened to catch up on list traffic today.
It looks like Catalin already covered everything, but just to sum up -- the patches need to be updated to avoid enabling the VDSO gettimeofday on systems where CNTVCT is not usable, for whatever reason. I do not intend to enable user space / VDSO access to CNTPCT to accommodate systems that don't initialize the virtual counter/offset correctly.
PATCH v10 4/6 ARM: add vdso user-space code
Quoted 7 lines · Nathan Lynch <Nathan_Lynch <at> mentor.com> writes:
Hi,
Sorry for not seeing this earlier but it doesn't look like my address was in the recipients list, and I just happened to catch up on list traffic today.
It looks like Catalin already covered everything, but just to sum up -
-
Quoted 5 lines
the patches need to be updated to avoid enabling the VDSO gettimeofday on systems where CNTVCT is not usable, for whatever reason. I do not intend to enable user space / VDSO access to CNTPCT to accommodate systems that don't initialize the virtual counter/offset correctly.
Hello Nathan,
Now, it's clear about current vDSO patch set and it is validated with virtual timer/VCT by using workaround with the below results on
ARMv7-A: CortexA7: Quad core, 1.2GHz,
About 1 iteration (gettimeofday),
WITHOUT VDSO : 30-35 ?sec
WITH VDSO : 4-7 ?sec
About 1000 iteration (gettimeofday),
WITHOUT VDSO : 27-30 msec
WITH VDSO : 13-15 msec
As you have mentioned about avoid enabling the VDSO gettimeofday on systems where CNTVCT is not usable, for whatever reason, please take care about this one in the updated patch list so that the things will be very clean.
Meanwhile, i will discuss further on "firmware is broken about CNTVOFF initialization zero + HYP disabled" in the following thread as mentioned in the previous reply to Catalin,
arm: Handle starting up in secure mode
https://lkml.org/lkml/2014/9/16/552
Thanks for the patch list,
BR,
Venkappa
PATCH v10 5/6 ARM: vdso initialization, mapping, and synchronization
In reply to Nathan LynchInitialize the vdso page list at boot, install the vdso mapping at exec time, and update the data page during timer ticks. This code is not built if CONFIG_VDSO is not enabled.
Account for the vdso length when randomizing the offset from the stack. The vdso is placed immediately following the sigpage with a separate install_special_mapping call in arm_install_vdso.
--- arch/arm/kernel/process.c | 17 +++- arch/arm/kernel/vdso.c | 211 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 225 insertions(+), 3 deletions(-) create mode 100644 arch/arm/kernel/vdso.c
arch/arm/kernel/process.c +14 -3
@@ -41,6 +41,7 @@#include <asm/system_misc.h> #include <asm/mach/time.h> #include <asm/tls.h> +#include <asm/vdso.h> #ifdef CONFIG_CC_STACKPROTECTOR #include <linux/stackprotector.h>
@@ -476,7 +477,7 @@ const char *arch_vma_name(struct vm_area_struct *vma) }/* If possible, provide a placement hint at a random offset from the -* stack for the signal page. +* stack for the sigpage and vdso pages. */ static unsigned long sigpage_addr (const struct mm_struct *mm,unsigned int npages)
@@ -520,6 +521,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) {struct mm_struct *mm = current->mm;struct vm_area_struct *vma; +unsigned long npages;unsigned long addr;unsigned long hint;int ret =0 ;
@@ -529,9 +531,12 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)if (!signal_page)return -ENOMEM ; + npages =1 ;/* for sigpage */ + npages += vdso_total_pages; +down_write (&mm->mmap_sem); - hint =sigpage_addr (mm,1 ); - addr =get_unmapped_area (NULL , hint,PAGE_SIZE ,0 ,0 ); + hint =sigpage_addr (mm, npages); + addr =get_unmapped_area (NULL , hint, npages <<PAGE_SHIFT ,0 ,0 );if (IS_ERR_VALUE (addr)) { ret = addr; goto up_fail;
@@ -548,6 +553,12 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) mm->context.sigpage = addr; +/* Unlike the sigpage, failure to install the vdso is unlikely +* to be fatal to the process, so no error check needed +* here. +*/ +arm_install_vdso (mm, addr +PAGE_SIZE ); + up_fail:up_write (&mm->mmap_sem);return ret;
arch/arm/kernel/vdso.c new file +211 -0
211 lines not on this page · raw message
Signature · 1 line
1.9.3
PATCH v10 6/6 ARM: add CONFIG_VDSO Kconfig and Makefile bits
In reply to Nathan LynchAllow users to enable the vdso in Kconfig; include the vdso in the build if CONFIG_VDSO is enabled. Add 'vdso_install' target.
--- arch/arm/Makefile | 8 ++++++++ arch/arm/kernel/Makefile | 1 + arch/arm/mm/Kconfig | 15 +++++++++++++++ 3 files changed, 24 insertions(+)
arch/arm/Makefile +8 -0
@@ -264,6 +264,7 @@ core-$(CONFIG_FPE_FASTFPE) += $(FASTFPE_OBJ) core-$(CONFIG_VFP) += arch/arm/vfp/ core-$(CONFIG_XEN) += arch/arm/xen/ core-$(CONFIG_KVM_ARM_HOST) += arch/arm/kvm/ +core-$(CONFIG_VDSO) += arch/arm/vdso/ # If we have a machine-specific directory, then include it in the build. core-y += arch/arm/kernel/ arch/arm/mm/ arch/arm/common/
@@ -316,6 +317,12 @@ PHONY += dtbs dtbs_install dtbs dtbs_install: prepare scripts $(Q)$(MAKE) $(build)=$(boot)/dts MACHINE=$(MACHINE) $@ +PHONY += vdso_install +vdso_install: +ifeq ($(CONFIG_VDSO),y) + $(Q)$(MAKE) $(build)=arch/arm/vdso $@ +endif + # We use MRPROPER_FILES and CLEAN_FILES now archclean: $(Q)$(MAKE) $(clean)=$(boot)
@@ -340,4 +347,5 @@ define archhelp echo ' Install using (your) ~/bin/$(INSTALLKERNEL) or' echo ' (distribution) /sbin/$(INSTALLKERNEL) or' echo ' install to $$(INSTALL_PATH) and run lilo' + echo ' vdso_install - Install unstripped vdso.so to $$(INSTALL_MOD_PATH)/vdso' endef
arch/arm/kernel/Makefile +1 -0
@@ -86,6 +86,7 @@ obj-$(CONFIG_PERF_EVENTS) += perf_regs.o obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o perf_event_cpu.o AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt obj-$(CONFIG_ARM_CPU_TOPOLOGY) += topology.o +obj-$(CONFIG_VDSO) += vdso.o ifneq ($(CONFIG_ARCH_EBSA110),y) obj-y += io.o
arch/arm/mm/Kconfig +15 -0
@@ -824,6 +824,21 @@ config KUSER_HELPERS Say N here only if you are absolutely certain that you do not need these helpers; otherwise, the safe option is to say Y. +config VDSO + bool "Enable vDSO for acceleration of some system calls" + depends on AEABI && MMU + default y if ARM_ARCH_TIMER + select GENERIC_TIME_VSYSCALL + help + Place in the process address space an ELF shared object + providing fast implementations of several system calls, + including gettimeofday and clock_gettime. Systems that + implement the ARM architected timer will receive maximum + benefit. + + You must have glibc 2.21 or later for programs to seamlessly + take advantage of this. + config DMA_CACHE_RWFO bool "Enable read/write for ownership DMA cache maintenance" depends on CPU_V6K && SMP
Signature · 1 line
1.9.3