Unify vDSOs across more architectures
Subsystems (11)
Vincenzo Frascino
maintainerv5 cover Unify vDSOs across more architectures
vDSO (virtual dynamic shared object) is a mechanism that the Linux kernel provides as an alternative to system calls to reduce where possible the costs in terms of cycles.
This is possible because certain syscalls like gettimeofday() do not write any data and return one or more values that are stored in the kernel, which makes relatively safe calling them directly as a library function.
Even if the mechanism is pretty much standard, every architecture in the last few years ended up implementing their own vDSO library in the architectural code.
The purpose of this patch-set is to identify the commonalities in between the architectures and try to consolidate the common code paths, starting with gettimeofday().
This implementation contains the following design choices:
* Every architecture defines the arch specific code in an header in "asm/vdso/".
* The generic implementation includes the arch specific one and lives in "lib/vdso".
* The arch specific code for gettimeofday lives in
"<arch path>/vdso/gettimeofday.c" and includes the generic code only.
* This approach allows to consolidate the common code in a single place with the benefit of avoiding code duplication.
This implementation contains the portings to the common library for: arm64, compat mode for arm64, arm, mips, x86_64, compat mode for x86_64 and i386.
The mips porting has been tested on qemu for mips32el. A configuration to repeat the tests can be found at [4].
The x86_64 porting has been tested on an Intel Xeon 5120T based machine running Ubuntu 18.04 and using the Ubuntu provided defconfig.
The i386 porting has been tested on qemu using the i386_defconfig configuration.
Last but not least from this porting arm64, compat arm64, arm and mips gain the support for:
* CLOCK_BOOTTIME that can be useful in certain scenarios since it keeps track of the time during sleep as well.
* CLOCK_TAI that is like CLOCK_REALTIME, but uses the International Atomic Time (TAI) reference instead of UTC to avoid jumping on leap second updates.
for both clock_gettime and clock_getres.
The porting has been validated using the vdsotest test-suite [1] extended to cover all the clock ids [2].
A new test has been added to the linux kselftest in order to validate the newly added library.
To simplify the testing, a copy of the patchset on top of a recent linux tree can be found at [3] and [4].
[1] https://github.com/nathanlynch/vdsotest
[2] https://github.com/fvincenzo/vdsotest
[3] git://linux-arm.org/linux-vf.git vdso/v5
[4] git://linux-arm.org/linux-vf.git vdso-mips/v5
Changes:
--------
v5:
- Rebased on 5.0-rc7.
- Added x86_64, compat mode for x86_64 and i386 portings.
- Extended vDSO kselftest.
- Addressed review comments.
v4:
- Rebased on 5.0-rc2.
- Addressed review comments.
- Disabled compat vdso on arm64 when the kernel is compiled with clang.
v3:
- Ported the latest fixes and optimizations done on the x86 architecture to the generic library.
- Addressed review comments.
- Improved the documentation of the interfaces.
- Changed the HAVE_ARCH_TIMER config option to a more generic HAVE_HW_COUNTER.
v2:
- Added -ffixed-x18 to arm64
- Repleced occurrences of timeval and timespec
- Modified datapage.h to be compliant with y2038 on all the architectures
- Removed __u_vdso type
Peter Collingbourne (1):
arm64: Build vDSO with -ffixed-x18
Vincenzo Frascino (22):
kernel: Standardize vdso_datapage
kernel: Define gettimeofday vdso common code
arm64: Substitute gettimeofday with C implementation
arm64: compat: Alloc separate pages for vectors and sigpage
arm64: compat: Split kuser32
arm64: compat: Refactor aarch32_alloc_vdso_pages()
arm64: compat: Add KUSER_HELPERS config option
arm64: compat: Add missing syscall numbers
arm64: compat: Expose signal related structures
arm64: compat: Generate asm offsets for signals
lib: vdso: Add compat support
arm64: compat: Add vDSO
arm64: Refactor vDSO code
arm64: compat: vDSO setup for compat layer
arm64: elf: vDSO code page discovery
arm64: compat: Get sigreturn trampolines from vDSO
arm64: Add vDSO compat support
arm64: Enable compat vDSO support
arm: Add support for generic vDSO
mips: Add support for generic vDSO
x86: Add support for generic vDSO
kselftest: Extend vDSO selftest
arch/arm/Kconfig | 3 +
arch/arm/include/asm/vdso/gettimeofday.h | 82 +++
arch/arm/include/asm/vdso_datapage.h | 25 +-
arch/arm/kernel/vdso.c | 108 ++--
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 +
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 +--------
arch/arm64/Kconfig | 24 +
arch/arm64/Makefile | 23 +-
arch/arm64/configs/defconfig | 1 +
arch/arm64/include/asm/elf.h | 20 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 58 +-
arch/arm64/include/asm/unistd.h | 3 +
arch/arm64/include/asm/vdso.h | 3 +
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++
.../include/asm/vdso/compat_gettimeofday.h | 119 ++++
arch/arm64/include/asm/vdso/gettimeofday.h | 85 +++
arch/arm64/include/asm/vdso_datapage.h | 47 --
arch/arm64/kernel/Makefile | 7 +-
arch/arm64/kernel/asm-offsets.c | 45 +-
arch/arm64/kernel/kuser32.S | 65 +--
arch/arm64/kernel/signal32.c | 76 +--
arch/arm64/kernel/sigreturn32.S | 46 ++
arch/arm64/kernel/vdso.c | 506 +++++++++++++-----
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 +
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 +++++++
arch/arm64/kernel/vdso32/note.c | 15 +
arch/arm64/kernel/vdso32/sigreturn.S | 62 +++
arch/arm64/kernel/vdso32/vdso.S | 19 +
arch/arm64/kernel/vdso32/vdso.lds.S | 81 +++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++
arch/mips/Kconfig | 2 +
arch/mips/include/asm/vdso.h | 60 +--
arch/mips/include/asm/vdso/gettimeofday.h | 165 ++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 107 +++-
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +
arch/x86/Kconfig | 3 +
arch/x86/entry/vdso/Makefile | 9 +
arch/x86/entry/vdso/vclock_gettime.c | 239 +--------
arch/x86/entry/vdso/vdso.lds.S | 2 +
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
arch/x86/entry/vdso/vdsox32.lds.S | 1 +
arch/x86/entry/vsyscall/vsyscall_gtod.c | 110 ++--
arch/x86/include/asm/vdso/gettimeofday.h | 203 +++++++
arch/x86/include/asm/vgtod.h | 73 +--
arch/x86/include/asm/vvar.h | 2 +-
include/vdso/datapage.h | 75 +++
include/vdso/helpers.h | 52 ++
include/vdso/types.h | 39 ++
lib/Kconfig | 5 +
lib/vdso/Kconfig | 37 ++
lib/vdso/Makefile | 22 +
lib/vdso/gettimeofday.c | 179 +++++++
tools/testing/selftests/vDSO/Makefile | 2 +
tools/testing/selftests/vDSO/vdso_full_test.c | 246 +++++++++
66 files changed, 2812 insertions(+), 1627 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
create mode 100644 arch/arm64/kernel/sigreturn32.S
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
create mode 100644 arch/x86/include/asm/vdso/gettimeofday.h
create mode 100644 include/vdso/datapage.h
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.cSignature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 1/23 kernel: Standardize vdso_datapage
In an effort to unify the common code for managing the vdso library in between all the architectures that support it, this patch tries to provide a common format for the vdso datapage.
As a result of this, this patch generalized the data structures in vgtod.h from x86 private includes to general includes (include/vdso).
--- include/vdso/datapage.h | 74 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 include/vdso/datapage.h
include/vdso/datapage.h new file +74 -0
@@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __VDSO_DATAPAGE_H +#define __VDSO_DATAPAGE_H + +#ifdef __KERNEL__ + +#ifndef __ASSEMBLY__ + +#include <linux/bits.h> +#include <linux/types.h> +#include <linux/time.h> + +#define VDSO_BASES (CLOCK_TAI + 1) +#define VDSO_HRES (BIT(CLOCK_REALTIME) | \ + BIT(CLOCK_MONOTONIC) | \ + BIT(CLOCK_MONOTONIC_RAW) | \ + BIT(CLOCK_BOOTTIME) | \ + BIT(CLOCK_TAI)) +#define VDSO_COARSE (BIT(CLOCK_REALTIME_COARSE) | \ + BIT(CLOCK_MONOTONIC_COARSE)) + +/* +* There is one vdso_timestamp object in vvar for each vDSO-accelerated +* clock_id. For high-resolution clocks, this encodes the time +* corresponding to vdso_data.cycle_last. For coarse clocks this encodes +* the actual time. +* +* To be noticed that for highres clocks nsec is left-shifted by +* vdso_data.cs[x].shift. +*/ +struct vdso_timestamp { +u64 sec; +u64 nsec; +}; + +#define CLOCKSOURCE_RAW 0 +#define CLOCKSOURCE_MONO 1 +#define CLOCKSOURCE_BASES (CLOCKSOURCE_MONO + 1) + +/* +* There is one vdso_clocksource object in vvar for each vDSO clocksource +* (mono, raw). This struct is designed to keep vdso_data "cache-line friendly" +* and optimal in terms of access pattern. +* +* Note that mask and shift are the same for mono and raw. +*/ +struct vdso_clocksource { +u64 mask;/* Clocksource mask */ +u32 mult;/* Clocksource multiplier */ +u32 shift;/* Clocksource shift */ +}; + +/* +* vdso_data will be accessed by 32 and 64 bit code at the same time +* so we should be careful before modifying this structure. +*/ +struct vdso_data { +u32 seq;/* Timebase sequence counter */ + +s32 clock_mode; +u64 cycle_last;/* Timebase at clocksource init */ + +struct vdso_clocksource cs[CLOCKSOURCE_BASES ]; +struct vdso_timestamp basetime[VDSO_BASES ]; + +s32 tz_minuteswest;/* Timezone definitions */ +s32 tz_dsttime; +}; + +#endif /* !__ASSEMBLY__ */ + +#endif /* __KERNEL__ */ + +#endif /* __VDSO_DATAPAGE_H */
Signature · 1 line
2.20.1
Mark Rutland
reviewerRe: [PATCH v5 01/23] kernel: Standardize vdso_datapage
Hi Vincenzo,
Quoted 1 line · Vincenzo Frascino
+#ifdef
__KERNEL__
I believe that __KERNEL__ guards haven't been necessary (in non-UAPI headers) since the UAPI split, and can be removed.
[...]
include/vdso/datapage.h line 11
Quoted 3 lines
include/vdso/datapage.h new file +3
-0+#include <linux/bits.h> +#include <linux/types.h> +#include <linux/time.h>
Nit: could we please keep these in alphabetical order?
Quoted 8 lines
+/*
+ * There is onevdso_timestampobject in vvar for each vDSO-accelerated
+ *clock_id. For high-resolution clocks, this encodes the time
+ * corresponding to vdso_data.cycle_last. For coarse clocks this encodes
+ * the actual time.
+ *
+ * To be noticed that for highres clocks nsec is left-shifted by
+ * vdso_data.cs[x].shift.
Nit: this would read better as:
Note that for highres clocks nsec is left-shifted by vdso_data.cs[x].shift.
[...]
Quoted 4 lines
+/*
+ *vdso_datawill be accessed by 32 and 64 bit code at the same time
+ * so we should be careful before modifying this structure.
+ */
Perhaps:
/*
* This structure will be accessed by native and compat vDSOs, so we
* need to ensure the layout is identical for native and compat code. */
... which I think is what you're trrying to say?
Thanks,
Mark.
Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
In reply to Vincenzo FrascinoReview
Quoted 11 lines · Vincenzo Frascino
(quoted diff) +11
-0+/* + * There is one vdso_clocksource object in vvar for each vDSO clocksource + * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly" + * and optimal in terms of access pattern. + * + * Note that mask and shift are the same for mono and raw. + */ +struct vdso_clocksource { + u64 mask; /* Clocksource mask */ + u32 mult; /* Clocksource multiplier */ + u32 shift; /* Clocksource shift */
Can you please get rid of the tail comments and use proper kerneldoc format?
Review
Quoted 11 lines
(quoted diff) +11
-0+/* + * vdso_data will be accessed by 32 and 64 bit code at the same time + * so we should be careful before modifying this structure. + */ +struct vdso_data { + u32 seq; /* Timebase sequence counter */ + + s32 clock_mode; + u64 cycle_last; /* Timebase at clocksource init */ + + struct vdso_clocksource cs[CLOCKSOURCE_BASES];
Why would you need different clocksource parameters? That really bloats the data structure and makes the cache access pattern worse. Also the vdso update needs to copy the same data over and over for no value.
The only clock ID which needs a different mult/shift would be MONOTONIC_RAW, but if we expose that through the VDSO then we really can be smarter than this. See incomplete and uncompilable patch below for reference. You get the idea.
Review
Quoted 4 lines
(quoted diff) +4
-0+ struct vdso_timestamp basetime[VDSO_BASES]; + + s32 tz_minuteswest; /* Timezone definitions */ + s32 tz_dsttime;
Also please keep the tabular alignment of the members
u32 foo; struct bar bar;
that's way better to read than
u32 foo; struct bar bar;
Thanks
tglx
8<-----------------
arch/x86/entry/vdso/vclock_gettime.c +11 -7
@@ -139,9 +139,10 @@ notrace static inline u64 vgetcyc(int moreturn U64_MAX ; } -notrace static int do_hres(clockid_t clk, struct timespec *ts) +notrace static int do_hres(clockid_t clk, struct timespec *ts, +struct vsyscall_gtod_data *vdata) { -struct vgtod_ts *base = >od->basetime[clk]; +struct vgtod_ts *base = &vdata->basetime[clk];u64 cycles, last, sec, ns;unsigned int seq;
@@ -168,9 +169,10 @@ notrace static int do_hres(clockid_t clkreturn 0 ; } -notrace static void do_coarse(clockid_t clk, struct timespec *ts) +notrace static void do_coarse(clockid_t clk, struct timespec *ts, +struct vsyscall_gtod_data *vdata) { -struct vgtod_ts *base = >od->basetime[clk]; +struct vgtod_ts *base = &vdata->basetime[clk];unsigned int seq; do {
@@ -194,10 +196,12 @@ notrace int __vdso_clock_gettime(clockid*/ msk =1U << clock;if (likely (msk &VGTOD_HRES )) { -return do_hres (clock, ts); +return do_hres (clock, ts, >od[0 ]); }else if (msk &VGTOD_COARSE ) { -do_coarse (clock, ts); +do_coarse (clock, ts, >od[0 ]);return 0 ; + }else if (msk &VGTOD_RAW ) { +return do_hres (clock, ts, >od[1 ]); }return vdso_fallback_gettime (clock, ts); }
@@ -210,7 +214,7 @@ notrace int __vdso_gettimeofday(struct tif (likely (tv !=NULL )) {struct timespec *ts = (struct timespec *) tv; -do_hres (CLOCK_REALTIME , ts); +do_hres (CLOCK_REALTIME , ts, >od[0 ]); tv->tv_usec /=1000 ; }if (unlikely (tz !=NULL )) {
arch/x86/include/asm/vgtod.h +3 -1
@@ -30,6 +30,7 @@ struct vgtod_ts {#define VGTOD_BASES (CLOCK_TAI + 1)#define VGTOD_HRES (BIT(CLOCK_REALTIME) | BIT(CLOCK_MONOTONIC) | BIT(CLOCK_TAI))#define VGTOD_COARSE (BIT(CLOCK_REALTIME_COARSE) | BIT(CLOCK_MONOTONIC_COARSE)) +#define VGTOD_RAW BIT(CLOCK_MOTONONIC_RAW)/* * vsyscall_gtod_data will be accessed by 32 and 64 bit code at the same time
@@ -49,7 +50,8 @@ struct vsyscall_gtod_data {int tz_minuteswest;int tz_dsttime; }; -extern struct vsyscall_gtod_data vsyscall_gtod_data; + +extern struct vsyscall_gtod_data vsyscall_gtod_data[2 ];extern int vclocks_used;static inline bool vclock_was_used (int vclock)
Vincenzo Frascino
maintainerRe: [PATCH v5 01/23] kernel: Standardize vdso_datapage
Quoted 16 lines · Thomas Gleixner
Review
Quoted 11 lines · Vincenzo Frascino
(quoted diff) +11
-0+/* + * There is one vdso_clocksource object in vvar for each vDSO clocksource + * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly" + * and optimal in terms of access pattern. + * + * Note that mask and shift are the same for mono and raw. + */ +struct vdso_clocksource { + u64 mask; /* Clocksource mask */ + u32 mult; /* Clocksource multiplier */ + u32 shift; /* Clocksource shift */Can you please get rid of the tail comments and use proper kerneldoc format?
I will fix it in v6, thanks.
Quoted 16 lines
Review
Quoted 11 lines
(quoted diff) +11
-0+/* + * vdso_data will be accessed by 32 and 64 bit code at the same time + * so we should be careful before modifying this structure. + */ +struct vdso_data { + u32 seq; /* Timebase sequence counter */ + + s32 clock_mode; + u64 cycle_last; /* Timebase at clocksource init */ + + struct vdso_clocksource cs[CLOCKSOURCE_BASES];Why would you need different clocksource parameters? That really bloats the data structure and makes the cache access pattern worse. Also the vdso update needs to copy the same data over and over for no value.
Quoted 5 lines
The only clock ID which needs a different mult/shift would be
MONOTONIC_RAW, but if we expose that through the VDSO then we really can be smarter than this. See incomplete and uncompilable patch below for reference. You get the idea.
Ok, thank you for providing the reference code. I will update my patches in v6 accordingly.
...
Signature · 2 lines
Regards, Vincenzo
Vincenzo Frascino
maintainerPATCH v5 2/23 kernel: Define gettimeofday vdso common code
In reply to Vincenzo FrascinoIn the last few years we assisted to an explosion of vdso implementations that mostly share similar code.
Try to unify the gettimeofday vdso implementation introducing lib/vdso. The code contained in this library can ideally be reused by all the architectures avoiding, where possible, code duplication.
--- include/vdso/datapage.h | 1 + include/vdso/helpers.h | 52 ++++++++++++ include/vdso/types.h | 39 +++++++++ lib/Kconfig | 5 ++ lib/vdso/Kconfig | 37 +++++++++ lib/vdso/Makefile | 22 +++++ lib/vdso/gettimeofday.c | 175 ++++++++++++++++++++++++++++++++++++++++ 7 files changed, 331 insertions(+) create mode 100644 include/vdso/helpers.h create mode 100644 include/vdso/types.h create mode 100644 lib/vdso/Kconfig create mode 100644 lib/vdso/Makefile create mode 100644 lib/vdso/gettimeofday.c
include/vdso/datapage.h +1 -0
@@ -9,6 +9,7 @@#include <linux/bits.h> #include <linux/types.h> #include <linux/time.h> +#include <vdso/types.h> #define VDSO_BASES (CLOCK_TAI + 1)#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
include/vdso/helpers.h new file +52 -0
52 lines not on this page · raw message
include/vdso/types.h new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __VDSO_TYPES_H +#define __VDSO_TYPES_H + +#ifdef __KERNEL__ + +#ifndef __ASSEMBLY__ + +#include <linux/types.h> +#include <linux/time.h> + +/* +* The definitions below are required to overcome the limitations +* of time_t on 32 bit architectures, which overflows in 2038. +* The new code should use the replacements based on time64_t and +* timespec64. +* +* The abstraction below will be updated once the migration to +* time64_t is complete. +*/ +#ifdef CONFIG_GENERIC_VDSO_32 +#define __vdso_timespec old_timespec32 +#define __vdso_timeval old_timeval32 +#else +#ifdef ENABLE_COMPAT_VDSO +#define __vdso_timespec old_timespec32 +#define __vdso_timeval old_timeval32 +#else +#define __vdso_timespec __kernel_timespec +#define __vdso_timeval __kernel_old_timeval +#endif /* CONFIG_COMPAT_VDSO */ +#endif /* CONFIG_GENERIC_VDSO_32 */ + + +#endif /* !__ASSEMBLY__ */ + +#endif /* __KERNEL__ */ + +#endif /* __VDSO_TYPES_H */
lib/Kconfig +5 -0
@@ -565,6 +565,11 @@ config OID_REGISTRY config UCS2_STRING tristate +# +# generic vdso +# +source "lib/vdso/Kconfig" + source "lib/fonts/Kconfig" config SG_SPLIT
lib/vdso/Kconfig new file +37 -0
@@ -0,0 +1,37 @@ +# SPDX-License-Identifier: GPL-2.0 + +config HAVE_GENERIC_VDSO + bool + default n + +if HAVE_GENERIC_VDSO + +config GENERIC_GETTIMEOFDAY + bool + help + This is a generic implementation of gettimeofday vdso. + Each architecture that enables this feature has to + provide the fallback implementation. + +config GENERIC_VDSO_32 + bool + depends on GENERIC_GETTIMEOFDAY && !64BIT + help + This config option helps to avoid possible performance issues + in 32 bit only architectures. + +config GENERIC_COMPAT_VDSO + bool + help + This config option enables the compat VDSO layer. + +config CROSS_COMPILE_COMPAT_VDSO + string "32 bit Toolchain prefix for compat vDSO" + default "" + depends on GENERIC_COMPAT_VDSO + help + Defines the cross-compiler prefix for compiling compat vDSO. + If a 64 bit compiler (i.e. x86_64) can compile the VDSO for + 32 bit, it does not need to define this parameter. + +endif
lib/vdso/Makefile new file +22 -0
@@ -0,0 +1,22 @@ +# SPDX-License-Identifier: GPL-2.0 + +GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST))) +GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH)) + +c-gettimeofday-$(CONFIG_GENERIC_GETTIMEOFDAY) := $(addprefix $(GENERIC_VDSO_DIR), gettimeofday.c) + +# This cmd checks that the vdso library does not contain absolute relocation +# It has to be called after the linking of the vdso library and requires it +# as a parameter. +# +# $(ARCH_REL_TYPE_ABS) is defined in the arch specific makefile and corresponds +# to the absolute relocation types printed by "objdump -R" and accepted by the +# dynamic linker. +ifndef ARCH_REL_TYPE_ABS +$(error ARCH_REL_TYPE_ABS is not set) +endif + +quiet_cmd_vdso_check = VDSOCHK $@ + cmd_vdso_check = if $(OBJDUMP) -R $@ | egrep -h "$(ARCH_REL_TYPE_ABS)"; \ + then (echo >&2 "$@: dynamic relocations are not supported"; \ + rm -f $@; /bin/false); fi
lib/vdso/gettimeofday.c new file +175 -0
175 lines not on this page · raw message
Signature · 1 line
2.20.1
Mark Rutland
reviewerRe: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
Quoted 42 lines · Vincenzo Frascino
In the last few years we assisted to an explosion of vdso implementations that mostly share similar code.
Try to unify the gettimeofday vdso implementation introducing lib/vdso. The code contained in this library can ideally be reused by all the architectures avoiding, where possible, code duplication.
Signed-off-by Vincenzo Frascino <[email protected]>--- include/vdso/datapage.h | 1 + include/vdso/helpers.h | 52 ++++++++++++ include/vdso/types.h | 39 +++++++++ lib/Kconfig | 5 ++ lib/vdso/Kconfig | 37 +++++++++ lib/vdso/Makefile | 22 +++++ lib/vdso/gettimeofday.c | 175 ++++++++++++++++++++++++++++++++++++++++ 7 files changed, 331 insertions(+) create mode 100644 include/vdso/helpers.h create mode 100644 include/vdso/types.h create mode 100644 lib/vdso/Kconfig create mode 100644 lib/vdso/Makefile create mode 100644 lib/vdso/gettimeofday.cinclude/vdso/datapage.h +1
-0@@ -9,6 +9,7 @@#include <linux/bits.h> #include <linux/types.h> #include <linux/time.h> +#include <vdso/types.h> #define VDSO_BASES (CLOCK_TAI + 1)#define VDSO_HRES (BIT(CLOCK_REALTIME) | \include/vdso/helpers.h new file +0
-0
include/vdso/helpers.h line 5
Quoted 5 lines
include/vdso/helpers.h new file +5
-0@@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __VDSO_HELPERS_H +#define __VDSO_HELPERS_H + +#ifdef __KERNEL__
Nit: __KERNEL__ guards can go.
include/vdso/helpers.h line 11
Quoted 6 lines
include/vdso/helpers.h new file +6
-0+ +#ifndef __ASSEMBLY__ + +#include <vdso/datapage.h> + +static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
Rather than explicitly annotating all functions with notrace, can't we disable instrumentation for all C files used for the vDSO using compiler flags?
That would be more robust, and make the code less noisy to read.
include/vdso/helpers.h line 24
Quoted 13 lines
include/vdso/helpers.h new file +13
-0+{ +u32 seq; + +repeat: + seq =READ_ONCE (vd->seq); +if (seq &1 ) { +cpu_relax (); + goto repeat; + } + +smp_rmb (); +return seq; +}
You could simplify the repeat loop as:
while ((seq = READ_ONCE(vd->seq)) & 1) cpu_relax();
Quoted 12 lines
include/vdso/helpers.h new file +12
-0+ +static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd, +u32 start) +{ +u32 seq; + +smp_rmb (); + seq =READ_ONCE (vd->seq); +return seq != start; +} + +static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
include/vdso/helpers.h line 46
Quoted 10 lines
include/vdso/helpers.h new file +10
-0+{ + ++vd->seq; +smp_wmb (); +} + +static __always_inline notrace void vdso_write_end(struct vdso_data *vd) +{ +smp_wmb (); + ++vd->seq; +}
I realise this is what existing vDSO update code does, but I do think that these should be using WRITE_ONCE() to perform the update, to ensure that the write is not torn.
e.g. these should be:
static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
{
WRITE_ONCE(vd->seq, vd->seq + 1);
smp_wmb();
}static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
{
smp_wmb();
WRITE_ONCE(vd->seq, vd->seq + 1);
}Otherwise, the compiler can validly tear updates to vd->seq, and there's the possibility that a reader sees an inconsistent value for vd->seq, and consequently uses inconsistent data read from the vdso data page.
[...]
include/vdso/helpers.h line 48
Quoted 2 lines
include/vdso/helpers.h new file +2
-0+#include <linux/types.h> +#include <linux/time.h>
Nit: please order headers alphabetically>
Quoted 12 lines
include/vdso/helpers.h new file +12
-0+ +/* +* The definitions below are required to overcome the limitations +* of time_t on 32 bit architectures, which overflows in 2038. +* The new code should use the replacements based on time64_t and +* timespec64. +* +* The abstraction below will be updated once the migration to +* time64_t is complete. +*/ +#ifdef CONFIG_GENERIC_VDSO_32 +#define __vdso_timespec old_timespec32
include/vdso/helpers.h
Quoted 10 lines
include/vdso/helpers.h new file +10
-0+#define __vdso_timeval old_timeval32 +#else +#ifdef ENABLE_COMPAT_VDSO +#define __vdso_timespec old_timespec32 +#define __vdso_timeval old_timeval32 +#else +#define __vdso_timespec __kernel_timespec +#define __vdso_timeval __kernel_old_timeval +#endif /* CONFIG_COMPAT_VDSO */ +#endif /* CONFIG_GENERIC_VDSO_32 */
I'm not sure what the comment is trying to say.
For a 64-bit kernel, does this affec the native vDSO, or just the compat vDSO?
[...]
include/vdso/helpers.h
Quoted 3 lines
include/vdso/helpers.h new file +3
-0+config HAVE_GENERIC_VDSO +bool + default n
IIRC, 'n' is the implicit default.
Thanks,
Mark.
Arnd Bergmann
maintainerRe: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
In reply to Vincenzo FrascinoQuoted 11 lines · Vincenzo Frascino
include/vdso/types.h new file +11
-0+/* +* The definitions below are required to overcome the limitations +* of time_t on 32 bit architectures, which overflows in 2038. +* The new code should use the replacements based on time64_t and +* timespec64. +* +* The abstraction below will be updated once the migration to +* time64_t is complete. +*/ +#ifdef CONFIG_GENERIC_VDSO_32 +#define __vdso_timespec old_timespec32
include/vdso/types.h line 32
Quoted 10 lines
include/vdso/types.h new file +10
-0+#define __vdso_timeval old_timeval32 +#else +#ifdef ENABLE_COMPAT_VDSO +#define __vdso_timespec old_timespec32 +#define __vdso_timeval old_timeval32 +#else +#define __vdso_timespec __kernel_timespec +#define __vdso_timeval __kernel_old_timeval +#endif /* CONFIG_COMPAT_VDSO */ +#endif /* CONFIG_GENERIC_VDSO_32 */
I don't think we need __vdso_timeval at all, just use __kernel_old_timeval everywhere. There is no generic 64-bit timeval type in the kernel, and there won't be because gettimeofday() is deprecated.
For __vdso_timespec, I see how you ended up with this redefinition, and it makes the current version of your patches easier, but I fear it will in turn make it harder to add the __kernel_old_timeval based variant.
What I'd prefer to see here is to always use the fixed length types everywhere in the code, such as
static notrace int __cvdso_clock_gettime64(clockid_t clock,
struct __kernel_timespec *ts)
{
... /* normal clock_gettime using 64-bit times */
}
static notrace int __cvdso_clock_gettime32(clockid_t clock,
struct old_timespec32 *ts32)
{
struct __kernel_timespec ts;
int ret = __cvdso_clock_gettime64(clock, &ts);
ts32->tv_sec = ts->tv_sec;
ts32->tv_nsec = ts->tv_nsec;
return ret;
}and then have two different versions for 32-bit and 64-bit architectures, calling one or the other function of the generic code.
Arnd
Arnd Bergmann
maintainerRe: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
Quoted 6 lines · Arnd Bergmann
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino <[email protected]> wrote:
For__vdso_timespec, I see how you ended up with this
redefinition, and it makes the current version of your patches
easier, but I fear it will in turn make it harder to add the__kernel_old_timevalbased variant.
I meant __kernel_timespec here, not __kernel_old_timeval.
Arnd
Vincenzo Frascino
maintainerRe: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
In reply to Arnd BergmannHi Arnd,
thank you for your review.
Quoted 21 lines · Arnd Bergmann
Quoted 18 lines · Vincenzo Frascino
include/vdso/types.h new file +18
-0+/* +* The definitions below are required to overcome the limitations +* of time_t on 32 bit architectures, which overflows in 2038. +* The new code should use the replacements based on time64_t and +* timespec64. +* +* The abstraction below will be updated once the migration to +* time64_t is complete. +*/ +#ifdef CONFIG_GENERIC_VDSO_32 +#define __vdso_timespec old_timespec32 +#define __vdso_timeval old_timeval32 +#else +#ifdef ENABLE_COMPAT_VDSO +#define __vdso_timespec old_timespec32 +#define __vdso_timeval old_timeval32 +#else +#define __vdso_timespec __kernel_timespec
Quoted 9 lines
include/vdso/types.h line 32
Quoted 3 lines
include/vdso/types.h new file +3
-0+#define __vdso_timeval __kernel_old_timeval +#endif /* CONFIG_COMPAT_VDSO */ +#endif /* CONFIG_GENERIC_VDSO_32 */ I don't think we need
__vdso_timevalat all, just use__kernel_old_timevaleverywhere. There is no generic 64-bit timeval type in the kernel, and there won't be becausegettimeofday()is deprecated.
Ok, I will update my implementation accordingly in v6.
Quoted 5 lines
For
__vdso_timespec, I see how you ended up with this redefinition, and it makes the current version of your patches easier, but I fear it will in turn make it harder to add the__kernel_old_timevalbased variant.
What is __kernel_old_timespec (based on you next email)? Why do you think it will make harder to add the new variants?
Signature · 2 lines
Regards, Vincenzo
Quoted 27 lines
What I'd prefer to see here is to always use the fixed length types everywhere in the code, such as
static notrace int __cvdso_clock_gettime64(clockid_t clock,
struct__kernel_timespec*ts)
{
... /* normalclock_gettimeusing 64-bit times */
}static notrace int __cvdso_clock_gettime32(clockid_t clock,
structold_timespec32*ts32){ struct __kernel_timespec ts; int ret = __cvdso_clock_gettime64(clock, &ts); ts32->tv_sec = ts->tv_sec; ts32->tv_nsec = ts->tv_nsec; return ret; }and then have two different versions for 32-bit and 64-bit architectures, calling one or the other function of the generic code.
Arnd
Arnd Bergmann
maintainerRe: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
Quoted 32 lines · Vincenzo Frascino
Quoted 21 lines · Arnd Bergmann
Quoted 18 lines · Vincenzo Frascino
include/vdso/types.h new file +18
-0+/* +* The definitions below are required to overcome the limitations +* of time_t on 32 bit architectures, which overflows in 2038. +* The new code should use the replacements based on time64_t and +* timespec64. +* +* The abstraction below will be updated once the migration to +* time64_t is complete. +*/ +#ifdef CONFIG_GENERIC_VDSO_32 +#define __vdso_timespec old_timespec32 +#define __vdso_timeval old_timeval32 +#else +#ifdef ENABLE_COMPAT_VDSO +#define __vdso_timespec old_timespec32 +#define __vdso_timeval old_timeval32 +#else +#define __vdso_timespec __kernel_timespecQuoted 9 lines
include/vdso/types.h line 32
Quoted 3 lines
include/vdso/types.h new file +3
-0+#define __vdso_timeval __kernel_old_timeval +#endif /* CONFIG_COMPAT_VDSO */ +#endif /* CONFIG_GENERIC_VDSO_32 */ For
__vdso_timespec, I see how you ended up with this redefinition, and it makes the current version of your patches easier, but I fear it will in turn make it harder to add the__kernel_old_timevalbased variant.
Quoted 2 lines
What is
__kernel_old_timespec(based on you next email)? Why do you think it will make harder to add the new variants?
I mean you should ideally use the types that you have listed above directly, without the abstraction.
In the long run I want one set of functions using __kernel_timespec that implements the 64-bit time interfaces on both 32-bit and 64-bit kernels, including the compat vdso, and another set using old_timespec32 for just the native 32-bit and compat version.
This would match what we do in the normal system calls (in linux-5.1+), where we always have the regular implementation use 64-bit types only, and have an optional _time32 version that is used for existing 32-bit user space, on both native 32 bit kernels and compat syscalls on 64-bit.
Arnd
Arnd Bergmann
maintainerPATCH 1/2 vdso: use fixed-size time types
Hi Vincento,
I've implemented now what I tried to explain earlier. This moves the implementation of clock_gettime() and clock_getres() over to 64-bit __kernel_timespec, with a conversion in the end for architectures that actually use the 32-bit calls.
The vdso/types.h file is now obsolete, but we do need to duplicate a few functions on mips32 and x86-32. It may be possible to fold those into the generic lib/vdso/gettimeofday.c file as another step.
If there are no objections, please fold these changes into your series.
--- arch/arm/include/asm/vdso/gettimeofday.h | 16 +++--- arch/arm/vdso/vgettimeofday.c | 10 ++-- .../include/asm/vdso/compat_gettimeofday.h | 16 +++--- arch/arm64/include/asm/vdso/gettimeofday.h | 12 ++-- arch/arm64/kernel/vdso/vgettimeofday.c | 6 +- arch/arm64/kernel/vdso32/vgettimeofday.c | 14 +++-- arch/mips/include/asm/vdso/gettimeofday.h | 22 ++++--- arch/mips/vdso/vgettimeofday.c | 28 ++++++++- arch/x86/entry/vdso/vclock_gettime.c | 57 +++++++++++++------ arch/x86/include/asm/vdso/gettimeofday.h | 16 +++--- include/vdso/datapage.h | 1 - include/vdso/types.h | 39 ------------- lib/vdso/gettimeofday.c | 48 +++++++++++----- 13 files changed, 164 insertions(+), 121 deletions(-) delete mode 100644 include/vdso/types.h
arch/arm/include/asm/vdso/gettimeofday.h +8 -8
@@ -14,11 +14,11 @@extern struct vdso_data *__get_datapage (void ); static __always_inline notrace int gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) { register struct timezone *tz asm("r1") = _tz; - register struct __vdso_timeval *tv asm("r0") = _tv; + register struct __kernel_old_timeval *tv asm("r0") = _tv; registerlong ret asm ("r0" ); register long nr asm("r7") = __NR_gettimeofday;
@@ -33,12 +33,12 @@ static __always_inline notrace int gettimeofday_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("r1") = _ts; + register struct __kernel_timespec *ts asm("r1") = _ts; register clockid_t clkid asm("r0") = _clkid; registerlong ret asm ("r0" ); - register long nr asm("r7") = __NR_clock_gettime; + register long nr asm("r7") = __NR_clock_gettime64; asmvolatile ( " swi #0\n"
@@ -51,12 +51,12 @@ static __always_inline notrace long clock_gettime_fallback( static __always_inline notrace int clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("r1") = _ts; + register struct __kernel_timespec *ts asm("r1") = _ts; register clockid_t clkid asm("r0") = _clkid; registerlong ret asm ("r0" ); - register long nr asm("r7") = __NR_clock_getres; + register long nr asm("r7") = __NR_clock_getres_time64; asmvolatile ( " swi #0\n"
arch/arm/vdso/vgettimeofday.c +5 -5
@@ -9,21 +9,21 @@#include <linux/types.h> notrace int __vdso_clock_gettime(clockid_t clock, -struct __vdso_timespec *ts) +struct old_timespec32 *ts) { -return __cvdso_clock_gettime (clock, ts); +return __cvdso_clock_gettime32 (clock, ts); } -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv, +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {return __cvdso_gettimeofday (tv, tz); } notrace int __vdso_clock_getres(clockid_t clock_id, -struct __vdso_timespec *res) +struct old_timespec32 *res) { -return __cvdso_clock_getres (clock_id, res); +return __cvdso_clock_getres_time32 (clock_id, res); }/* Avoid unresolved references emitted by GCC */
arch/arm64/include/asm/vdso/compat_gettimeofday.h +8 -8
@@ -24,11 +24,11 @@ extern struct vdso_data _vdso_data __attribute__((visibility("hidden"))); static __always_inline notrace int gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) { register struct timezone *tz asm("r1") = _tz; - register struct __vdso_timeval *tv asm("r0") = _tv; + register struct __kernel_old_timeval *tv asm("r0") = _tv; registerlong ret asm ("r0" ); register long nr asm("r7") = __NR_compat_gettimeofday;
@@ -43,12 +43,12 @@ static __always_inline notrace int gettimeofday_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("r1") = _ts; + register struct __kernel_timespec *ts asm("r1") = _ts; register clockid_t clkid asm("r0") = _clkid; registerlong ret asm ("r0" ); - register long nr asm("r7") = __NR_compat_clock_gettime; + register long nr asm("r7") = __NR_compat_clock_gettime64; asmvolatile ( " swi #0\n"
@@ -61,12 +61,12 @@ static __always_inline notrace long clock_gettime_fallback( static __always_inline notrace int clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("r1") = _ts; + register struct __kernel_timespec *ts asm("r1") = _ts; register clockid_t clkid asm("r0") = _clkid; registerlong ret asm ("r0" ); - register long nr asm("r7") = __NR_compat_clock_getres; + register long nr asm("r7") = __NR_compat_clock_getres_time64;/* The checks below are required for ABI consistency with arm */ if ((_clkid >=MAX_CLOCKS ) || (_ts ==NULL ))
arch/arm64/include/asm/vdso/gettimeofday.h +6 -6
@@ -13,11 +13,11 @@extern struct vdso_data _vdso_data; static __always_inline notrace int gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) { register struct timezone *tz asm("x1") = _tz; - register struct __vdso_timeval *tv asm("x0") = _tv; + register struct __kernel_old_timeval *tv asm("x0") = _tv; registerlong ret asm ("x0" ); register long nr asm("x8") = __NR_gettimeofday;
@@ -32,9 +32,9 @@ static __always_inline notrace int gettimeofday_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("x1") = _ts; + register struct __kernel_timespec *ts asm("x1") = _ts; register clockid_t clkid asm("x0") = _clkid; registerlong ret asm ("x0" ); register long nr asm("x8") = __NR_clock_gettime;
@@ -50,9 +50,9 @@ static __always_inline notrace long clock_gettime_fallback( static __always_inline notrace int clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("x1") = _ts; + register struct __kernel_timespec *ts asm("x1") = _ts; register clockid_t clkid asm("x0") = _clkid; registerlong ret asm ("x0" ); register long nr asm("x8") = __NR_clock_getres;
arch/arm64/kernel/vdso/vgettimeofday.c +3 -3
@@ -9,19 +9,19 @@#include <linux/types.h> notrace int __kernel_clock_gettime(clockid_t clock, -struct __vdso_timespec *ts) +struct __kernel_timespec *ts) {return __cvdso_clock_gettime (clock, ts); } -notrace int __kernel_gettimeofday(struct __vdso_timeval *tv, +notrace int __kernel_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {return __cvdso_gettimeofday (tv, tz); } notrace int __kernel_clock_getres(clockid_t clock_id, -struct __vdso_timespec *res) +struct __kernel_timespec *res) {return __cvdso_clock_getres (clock_id, res); }
arch/arm64/kernel/vdso32/vgettimeofday.c +10 -4
@@ -9,21 +9,27 @@#include <linux/types.h> notrace int __vdso_clock_gettime(clockid_t clock, -struct __vdso_timespec *ts) +struct old_timespec32 *ts) +{ +return __cvdso_clock_gettime32 (clock, ts); +} + +notrace int __vdso_clock_gettime64(clockid_t clock, +struct __kernel_timespec *ts) {return __cvdso_clock_gettime (clock, ts); } -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv, +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {return __cvdso_gettimeofday (tv, tz); } notrace int __vdso_clock_getres(clockid_t clock_id, -struct __vdso_timespec *res) +struct old_timespec32 *res) { -return __cvdso_clock_getres (clock_id, res); +return __cvdso_clock_getres_time32 (clock_id, res); }/* Avoid unresolved references emitted by GCC */
arch/mips/include/asm/vdso/gettimeofday.h +15 -7
@@ -25,11 +25,11 @@#ifdef CONFIG_MIPS_CLOCK_VSYSCALL static __always_inline notrace long gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) { register struct timezone *tz asm("a1") = _tz; - register struct __vdso_timeval *tv asm("a0") = _tv; + register struct __kernel_old_timeval *tv asm("a0") = _tv; registerlong ret asm("v0" ); register long nr asm("v0") = __NR_gettimeofday; registerlong error asm("a3" );
@@ -47,7 +47,7 @@ static __always_inline notrace long gettimeofday_fallback(#else static __always_inline notrace long gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) {return -1 ;
@@ -57,12 +57,16 @@ static __always_inline notrace long gettimeofday_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("a1") = _ts; + register struct __kernel_timespec *ts asm("a1") = _ts; register clockid_t clkid asm("a0") = _clkid; registerlong ret asm("v0" ); +#if _MIPS_SIM == _MIPS_SIM_ABI64 register long nr asm("v0") = __NR_clock_gettime; +#else + register long nr asm("v0") = __NR_clock_gettime64 +#endif registerlong error asm("a3" ); asm volatile(
@@ -77,12 +81,16 @@ static __always_inline notrace long clock_gettime_fallback( static __always_inline notrace int clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("a1") = _ts; + register struct __kernel_timespec *ts asm("a1") = _ts; register clockid_t clkid asm("a0") = _clkid; registerlong ret asm("v0" ); +#if _MIPS_SIM == _MIPS_SIM_ABI64 register long nr asm("v0") = __NR_clock_getres; +#else + register long nr asm("v0") = __NR_clock_getres64 +#endif registerlong error asm("a3" ); asm volatile(
arch/mips/vdso/vgettimeofday.c +25 -3
@@ -8,21 +8,43 @@#include <linux/time.h> #include <linux/types.h> +#if _MIPS_SIM != _MIPS_SIM_ABI64 notrace int __vdso_clock_gettime(clockid_t clock, -struct __vdso_timespec *ts) +struct old_timespec32 *ts) +{ +return __cvdso_clock_gettime32 (clock, ts); +} + +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv, +struct timezone *tz) +{ +return __cvdso_gettimeofday_time32 (tv, tz); +} + +notrace int __vdso_clock_getres(clockid_t clock_id, +struct old_timespec32 *res) +{ +return __cvdso_clock_getres_time32 (clock_id, res); +} + +#else + +notrace int __vdso_clock_gettime(clockid_t clock, +struct __kernel_timespec *ts) {return __cvdso_clock_gettime (clock, ts); } -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv, +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {return __cvdso_gettimeofday (tv, tz); } notrace int __vdso_clock_getres(clockid_t clock_id, -struct __vdso_timespec *res) +struct __kernel_timespec *res) {return __cvdso_clock_getres (clock_id, res); } +#endif
arch/x86/entry/vdso/vclock_gettime.c +41 -16
76 lines not on this page · raw message
arch/x86/include/asm/vdso/gettimeofday.h +8 -8
@@ -38,7 +38,7 @@ extern u8 hvclock_page static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) {long ret; asm ("syscall" : "=a" (ret), "=m" (*_ts) :
@@ -48,7 +48,7 @@ static __always_inline notrace long clock_gettime_fallback( } static __always_inline notrace long gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) {long ret;
@@ -59,7 +59,7 @@ static __always_inline notrace long gettimeofday_fallback( static __always_inline notrace long clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) {long ret; asm ("syscall" : "=a" (ret), "=m" (*_ts) :
@@ -72,7 +72,7 @@ static __always_inline notrace long clock_getres_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) {long ret;
@@ -82,13 +82,13 @@ static __always_inline notrace long clock_gettime_fallback( "call __kernel_vsyscall \n" "mov %%edx, %%ebx \n" : "=a" (ret), "=m" (*_ts) - : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts) + : "0" (__NR_clock_gettime64), [clock] "g" (_clkid), "c" (_ts) : "edx");return ret; } static __always_inline notrace long gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) {long ret;
@@ -105,7 +105,7 @@ static __always_inline notrace long gettimeofday_fallback( static __always_inline notrace long clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) {long ret;
@@ -115,7 +115,7 @@ static __always_inline notrace long clock_getres_fallback( "call __kernel_vsyscall \n" "mov %%edx, %%ebx \n" : "=a" (ret), "=m" (*_ts) - : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts) + : "0" (__NR_clock_getres_time64), [clock] "g" (_clkid), "c" (_ts) : "edx");return ret; }
include/vdso/datapage.h +0 -1
@@ -9,7 +9,6 @@#include <linux/bits.h> #include <linux/types.h> #include <linux/time.h> -#include <vdso/types.h> #define VDSO_BASES (CLOCK_TAI + 1)#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
include/vdso/types.h deleted +0 -39
@@ -1,39 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __VDSO_TYPES_H -#define __VDSO_TYPES_H - -#ifdef __KERNEL__ - -#ifndef __ASSEMBLY__ - -#include <linux/types.h> -#include <linux/time.h> - -/* -* The definitions below are required to overcome the limitations -* of time_t on 32 bit architectures, which overflows in 2038. -* The new code should use the replacements based on time64_t and -* timespec64. -* -* The abstraction below will be updated once the migration to -* time64_t is complete. -*/ -#ifdef CONFIG_GENERIC_VDSO_32 -#define __vdso_timespec old_timespec32 -#define __vdso_timeval old_timeval32 -#else -#ifdef ENABLE_COMPAT_VDSO -#define __vdso_timespec old_timespec32 -#define __vdso_timeval old_timeval32 -#else -#define __vdso_timespec __kernel_timespec -#define __vdso_timeval __kernel_old_timeval -#endif /* CONFIG_COMPAT_VDSO */ -#endif /* CONFIG_GENERIC_VDSO_32 */ - - -#endif /* !__ASSEMBLY__ */ - -#endif /* __KERNEL__ */ - -#endif /* __VDSO_TYPES_H */
lib/vdso/gettimeofday.c +35 -13
97 lines not on this page · raw message
Signature · 1 line
2.20.0
Arnd Bergmann
maintainerPATCH 2/2 vdso: add clock_gettime64
Adding a clock_gettime64() vdso call on all32-bit machines is now trivial.
--- arch/arm/vdso/vdso.lds.S | 1 + arch/arm/vdso/vgettimeofday.c | 6 ++++++ arch/arm64/kernel/vdso32/vdso.lds.S | 1 + arch/mips/vdso/vdso.lds.S | 3 +++ arch/mips/vdso/vgettimeofday.c | 6 ++++++ arch/x86/entry/vdso/vclock_gettime.c | 6 ++++++ arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 + 7 files changed, 24 insertions(+)
arch/arm/vdso/vdso.lds.S +1 -0
@@ -83,6 +83,7 @@ VERSION __vdso_clock_gettime; __vdso_gettimeofday; __vdso_clock_getres; + __vdso_clock_gettime64; local: *; }; }
arch/arm/vdso/vgettimeofday.c +6 -0
@@ -14,6 +14,12 @@ notrace int __vdso_clock_gettime(clockid_t clock,return __cvdso_clock_gettime32 (clock, ts); } +notrace int __vdso_clock_gettime64(clockid_t clock, +struct __kernel_timespec *ts) +{ +return __cvdso_clock_gettime (clock, ts); +} + notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {
arch/arm64/kernel/vdso32/vdso.lds.S +1 -0
@@ -68,6 +68,7 @@ VERSION __kernel_sigreturn_thumb; __kernel_rt_sigreturn_arm; __kernel_rt_sigreturn_thumb; + __vdso_clock_gettime64; local: *; }; }
arch/mips/vdso/vdso.lds.S +3 -0
@@ -100,6 +100,9 @@ VERSION __vdso_clock_gettime; __vdso_gettimeofday; __vdso_clock_getres; +#if _MIPS_SIM != _MIPS_SIM_ABI64 + __vdso_clock_gettime64; +#endif #endif local: *; };
arch/mips/vdso/vgettimeofday.c +6 -0
@@ -27,6 +27,12 @@ notrace int __vdso_clock_getres(clockid_t clock_id,return __cvdso_clock_getres_time32 (clock_id, res); } +notrace int __vdso_clock_gettime_time64(clockid_t clock, +struct __kernel_timespec *ts) +{ +return __cvdso_clock_gettime (clock, ts); +} +#else notrace int __vdso_clock_gettime(clockid_t clock,
arch/x86/entry/vdso/vclock_gettime.c +6 -0
@@ -54,6 +54,7 @@ int clock_getres(clockid_t, struct __kernel_timespec *)#else /* i386 only */ extern int __vdso_clock_gettime (clockid_t clock,struct old_timespec32 *ts); +extern int __vdso_clock_gettime64 (clockid_t clock,struct __kernel_timespec *ts);extern int __vdso_clock_getres (clockid_t clock,struct old_timespec32 *res); notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
@@ -64,6 +65,11 @@ notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)int clock_gettime (clockid_t ,struct old_timespec32 *) __attribute__((weak,alias ("__vdso_clock_gettime" ))); +notrace int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts) +{ +return __cvdso_clock_gettime (clock, ts); +} + notrace int __vdso_clock_getres(clockid_t clock,struct old_timespec32 *res) {
arch/x86/entry/vdso/vdso32/vdso32.lds.S +1 -0
@@ -27,6 +27,7 @@ VERSION __vdso_gettimeofday; __vdso_time; __vdso_clock_getres; + __vdso_clock_gettime64; }; LINUX_2.5 {
Signature · 1 line
2.20.0
Re: [PATCH 1/2] vdso: use fixed-size time types
In reply to Arnd BergmannQuoted 7 lines · Arnd Bergmann
Hi Vincento,
I've implemented now what I tried to explain earlier. This moves the implementation of
clock_gettime()andclock_getres()over to 64-bit__kernel_timespec, with a conversion in the end for architectures that actually use the 32-bit calls.
Quoted 6 lines
The
vdso/types.hfile is now obsolete, but we do need to duplicate a few functions on mips32 and x86-32. It may be possible to fold those into the genericlib/vdso/gettimeofday.cfile as another step.If there are no objections, please fold these changes into your series.
Please write the new generic code in that way and apply the architecture changes when converting them over.
Thanks,
tglx
Vincenzo Frascino
maintainerRe: [PATCH 1/2] vdso: use fixed-size time types
In reply to Arnd BergmannHi Arnd,
thank you for providing the changes.
Quoted 7 lines · Arnd Bergmann
Hi Vincento,
I've implemented now what I tried to explain earlier. This moves the implementation of
clock_gettime()andclock_getres()over to 64-bit__kernel_timespec, with a conversion in the end for architectures that actually use the 32-bit calls.
Quoted 7 lines
The
vdso/types.hfile is now obsolete, but we do need to duplicate a few functions on mips32 and x86-32. It may be possible to fold those into the genericlib/vdso/gettimeofday.cfile as another step.If there are no objections, please fold these changes into your series.
No objections on my side, I will fold the changes in my set.
Quoted 692 lines
Signed-off-by Arnd Bergmann <[email protected]>--- arch/arm/include/asm/vdso/gettimeofday.h | 16 +++--- arch/arm/vdso/vgettimeofday.c | 10 ++-- .../include/asm/vdso/compat_gettimeofday.h | 16 +++--- arch/arm64/include/asm/vdso/gettimeofday.h | 12 ++-- arch/arm64/kernel/vdso/vgettimeofday.c | 6 +- arch/arm64/kernel/vdso32/vgettimeofday.c | 14 +++-- arch/mips/include/asm/vdso/gettimeofday.h | 22 ++++--- arch/mips/vdso/vgettimeofday.c | 28 ++++++++- arch/x86/entry/vdso/vclock_gettime.c | 57 +++++++++++++------ arch/x86/include/asm/vdso/gettimeofday.h | 16 +++--- include/vdso/datapage.h | 1 - include/vdso/types.h | 39 ------------- lib/vdso/gettimeofday.c | 48 +++++++++++----- 13 files changed, 164 insertions(+), 121 deletions(-) delete mode 100644 include/vdso/types.harch/arm/include/asm/vdso/gettimeofday.h +8
-8@@ -14,11 +14,11 @@extern struct vdso_data *__get_datapage (void ); static __always_inline notrace int gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) { register struct timezone *tz asm("r1") = _tz; - register struct __vdso_timeval *tv asm("r0") = _tv; + register struct __kernel_old_timeval *tv asm("r0") = _tv; registerlong ret asm ("r0" ); register long nr asm("r7") = __NR_gettimeofday;@@ -33,12 +33,12 @@ static __always_inline notrace int gettimeofday_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("r1") = _ts; + register struct __kernel_timespec *ts asm("r1") = _ts; register clockid_t clkid asm("r0") = _clkid; registerlong ret asm ("r0" ); - register long nr asm("r7") = __NR_clock_gettime; + register long nr asm("r7") = __NR_clock_gettime64; asmvolatile ( " swi #0\n"@@ -51,12 +51,12 @@ static __always_inline notrace long clock_gettime_fallback( static __always_inline notrace int clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("r1") = _ts; + register struct __kernel_timespec *ts asm("r1") = _ts; register clockid_t clkid asm("r0") = _clkid; registerlong ret asm ("r0" ); - register long nr asm("r7") = __NR_clock_getres; + register long nr asm("r7") = __NR_clock_getres_time64; asmvolatile ( " swi #0\n"arch/arm/vdso/vgettimeofday.c +5
-5@@ -9,21 +9,21 @@#include <linux/types.h> notrace int __vdso_clock_gettime(clockid_t clock, -struct __vdso_timespec *ts) +struct old_timespec32 *ts) { -return __cvdso_clock_gettime (clock, ts); +return __cvdso_clock_gettime32 (clock, ts); } -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv, +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {return __cvdso_gettimeofday (tv, tz); } notrace int __vdso_clock_getres(clockid_t clock_id, -struct __vdso_timespec *res) +struct old_timespec32 *res) { -return __cvdso_clock_getres (clock_id, res); +return __cvdso_clock_getres_time32 (clock_id, res); }/* Avoid unresolved references emitted by GCC */ arch/arm64/include/asm/vdso/compat_gettimeofday.h +8
-8@@ -24,11 +24,11 @@ extern struct vdso_data _vdso_data __attribute__((visibility("hidden"))); static __always_inline notrace int gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) { register struct timezone *tz asm("r1") = _tz; - register struct __vdso_timeval *tv asm("r0") = _tv; + register struct __kernel_old_timeval *tv asm("r0") = _tv; registerlong ret asm ("r0" ); register long nr asm("r7") = __NR_compat_gettimeofday;@@ -43,12 +43,12 @@ static __always_inline notrace int gettimeofday_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("r1") = _ts; + register struct __kernel_timespec *ts asm("r1") = _ts; register clockid_t clkid asm("r0") = _clkid; registerlong ret asm ("r0" ); - register long nr asm("r7") = __NR_compat_clock_gettime; + register long nr asm("r7") = __NR_compat_clock_gettime64; asmvolatile ( " swi #0\n"@@ -61,12 +61,12 @@ static __always_inline notrace long clock_gettime_fallback( static __always_inline notrace int clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("r1") = _ts; + register struct __kernel_timespec *ts asm("r1") = _ts; register clockid_t clkid asm("r0") = _clkid; registerlong ret asm ("r0" ); - register long nr asm("r7") = __NR_compat_clock_getres; + register long nr asm("r7") = __NR_compat_clock_getres_time64;/* The checks below are required for ABI consistency with arm */ if ((_clkid >=MAX_CLOCKS ) || (_ts ==NULL ))arch/arm64/include/asm/vdso/gettimeofday.h +6
-6@@ -13,11 +13,11 @@extern struct vdso_data _vdso_data; static __always_inline notrace int gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) { register struct timezone *tz asm("x1") = _tz; - register struct __vdso_timeval *tv asm("x0") = _tv; + register struct __kernel_old_timeval *tv asm("x0") = _tv; registerlong ret asm ("x0" ); register long nr asm("x8") = __NR_gettimeofday;@@ -32,9 +32,9 @@ static __always_inline notrace int gettimeofday_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("x1") = _ts; + register struct __kernel_timespec *ts asm("x1") = _ts; register clockid_t clkid asm("x0") = _clkid; registerlong ret asm ("x0" ); register long nr asm("x8") = __NR_clock_gettime;@@ -50,9 +50,9 @@ static __always_inline notrace long clock_gettime_fallback( static __always_inline notrace int clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("x1") = _ts; + register struct __kernel_timespec *ts asm("x1") = _ts; register clockid_t clkid asm("x0") = _clkid; registerlong ret asm ("x0" ); register long nr asm("x8") = __NR_clock_getres;arch/arm64/kernel/vdso/vgettimeofday.c +3
-3@@ -9,19 +9,19 @@#include <linux/types.h> notrace int __kernel_clock_gettime(clockid_t clock, -struct __vdso_timespec *ts) +struct __kernel_timespec *ts) {return __cvdso_clock_gettime (clock, ts); } -notrace int __kernel_gettimeofday(struct __vdso_timeval *tv, +notrace int __kernel_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {return __cvdso_gettimeofday (tv, tz); } notrace int __kernel_clock_getres(clockid_t clock_id, -struct __vdso_timespec *res) +struct __kernel_timespec *res) {return __cvdso_clock_getres (clock_id, res); }arch/arm64/kernel/vdso32/vgettimeofday.c +10
-4@@ -9,21 +9,27 @@#include <linux/types.h> notrace int __vdso_clock_gettime(clockid_t clock, -struct __vdso_timespec *ts) +struct old_timespec32 *ts) +{ +return __cvdso_clock_gettime32 (clock, ts); +} + +notrace int __vdso_clock_gettime64(clockid_t clock, +struct __kernel_timespec *ts) {return __cvdso_clock_gettime (clock, ts); } -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv, +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {return __cvdso_gettimeofday (tv, tz); } notrace int __vdso_clock_getres(clockid_t clock_id, -struct __vdso_timespec *res) +struct old_timespec32 *res) { -return __cvdso_clock_getres (clock_id, res); +return __cvdso_clock_getres_time32 (clock_id, res); }/* Avoid unresolved references emitted by GCC */ arch/mips/include/asm/vdso/gettimeofday.h +15
-7@@ -25,11 +25,11 @@#ifdef CONFIG_MIPS_CLOCK_VSYSCALL static __always_inline notrace long gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) { register struct timezone *tz asm("a1") = _tz; - register struct __vdso_timeval *tv asm("a0") = _tv; + register struct __kernel_old_timeval *tv asm("a0") = _tv; registerlong ret asm("v0" ); register long nr asm("v0") = __NR_gettimeofday; registerlong error asm("a3" );@@ -47,7 +47,7 @@ static __always_inline notrace long gettimeofday_fallback(#else static __always_inline notrace long gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) {return -1 ;@@ -57,12 +57,16 @@ static __always_inline notrace long gettimeofday_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("a1") = _ts; + register struct __kernel_timespec *ts asm("a1") = _ts; register clockid_t clkid asm("a0") = _clkid; registerlong ret asm("v0" ); +#if _MIPS_SIM == _MIPS_SIM_ABI64 register long nr asm("v0") = __NR_clock_gettime; +#else + register long nr asm("v0") = __NR_clock_gettime64 +#endif registerlong error asm("a3" ); asm volatile(@@ -77,12 +81,16 @@ static __always_inline notrace long clock_gettime_fallback( static __always_inline notrace int clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) { - register struct __vdso_timespec *ts asm("a1") = _ts; + register struct __kernel_timespec *ts asm("a1") = _ts; register clockid_t clkid asm("a0") = _clkid; registerlong ret asm("v0" ); +#if _MIPS_SIM == _MIPS_SIM_ABI64 register long nr asm("v0") = __NR_clock_getres; +#else + register long nr asm("v0") = __NR_clock_getres64 +#endif registerlong error asm("a3" ); asm volatile(arch/mips/vdso/vgettimeofday.c +25
-3@@ -8,21 +8,43 @@#include <linux/time.h> #include <linux/types.h> +#if _MIPS_SIM != _MIPS_SIM_ABI64 notrace int __vdso_clock_gettime(clockid_t clock, -struct __vdso_timespec *ts) +struct old_timespec32 *ts) +{ +return __cvdso_clock_gettime32 (clock, ts); +} + +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv, +struct timezone *tz) +{ +return __cvdso_gettimeofday_time32 (tv, tz); +} + +notrace int __vdso_clock_getres(clockid_t clock_id, +struct old_timespec32 *res) +{ +return __cvdso_clock_getres_time32 (clock_id, res); +} + +#else + +notrace int __vdso_clock_gettime(clockid_t clock, +struct __kernel_timespec *ts) {return __cvdso_clock_gettime (clock, ts); } -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv, +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {return __cvdso_gettimeofday (tv, tz); } notrace int __vdso_clock_getres(clockid_t clock_id, -struct __vdso_timespec *res) +struct __kernel_timespec *res) {return __cvdso_clock_getres (clock_id, res); } +#endif arch/x86/entry/vdso/vclock_gettime.c +41
-16@@ -12,25 +12,14 @@#include "../../../../lib/vdso/gettimeofday.c" -extern int __vdso_clock_gettime (clockid_t clock,struct __vdso_timespec *ts); -extern int __vdso_gettimeofday (struct __vdso_timeval *tv,struct timezone *tz); +extern int __vdso_gettimeofday (struct __kernel_old_timeval *tv,struct timezone *tz);extern time_t __vdso_time (time_t *t); -extern int __vdso_clock_getres (clockid_t clock,struct __vdso_timespec *res); - -notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts) -{ -return __cvdso_clock_gettime (clock, ts); -} - -int clock_gettime (clockid_t ,struct __vdso_timespec *) - __attribute__((weak,alias ("__vdso_clock_gettime" ))); - -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv, +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,struct timezone *tz) {return __cvdso_gettimeofday (tv, tz); } -int gettimeofday (struct __vdso_timeval *,struct timezone *) +int gettimeofday (struct __kernel_old_timeval *,struct timezone *) __attribute__((weak,alias ("__vdso_gettimeofday" ))); notrace time_t __vdso_time(time_t *t)@@ -40,10 +29,46 @@ notrace time_t __vdso_time(time_t *t)time_t time (time_t *t) __attribute__((weak,alias ("__vdso_time" ))); + +#ifdef __x86_64__ +/* both 64-bit and x32 use these */ +extern int __vdso_clock_gettime (clockid_t clock,struct __kernel_timespec *ts); +extern int __vdso_clock_getres (clockid_t clock,struct __kernel_timespec *res); + +notrace int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts) +{ +return __cvdso_clock_gettime (clock, ts); +} + +int clock_gettime (clockid_t ,struct __kernel_timespec *) + __attribute__((weak,alias ("__vdso_clock_gettime" ))); + notrace int __vdso_clock_getres(clockid_t clock, -struct __vdso_timespec *res) +struct __kernel_timespec *res) {return __cvdso_clock_getres (clock, res); } -int clock_getres (clockid_t ,struct __vdso_timespec *) +int clock_getres (clockid_t ,struct __kernel_timespec *) + __attribute__((weak,alias ("__vdso_clock_getres" ))); + +#else +/* i386 only */ +extern int __vdso_clock_gettime (clockid_t clock,struct old_timespec32 *ts); +extern int __vdso_clock_getres (clockid_t clock,struct old_timespec32 *res); + +notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts) +{ +return __cvdso_clock_gettime32 (clock, ts); +} + +int clock_gettime (clockid_t ,struct old_timespec32 *) + __attribute__((weak,alias ("__vdso_clock_gettime" ))); + +notrace int __vdso_clock_getres(clockid_t clock, +struct old_timespec32 *res) +{ +return __cvdso_clock_getres_time32 (clock, res); +} +int clock_getres (clockid_t ,struct old_timespec32 *) __attribute__((weak,alias ("__vdso_clock_getres" ))); +#endif arch/x86/include/asm/vdso/gettimeofday.h +8
-8@@ -38,7 +38,7 @@ extern u8 hvclock_page static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) {long ret; asm ("syscall" : "=a" (ret), "=m" (*_ts) :@@ -48,7 +48,7 @@ static __always_inline notrace long clock_gettime_fallback( } static __always_inline notrace long gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) {long ret;@@ -59,7 +59,7 @@ static __always_inline notrace long gettimeofday_fallback( static __always_inline notrace long clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) {long ret; asm ("syscall" : "=a" (ret), "=m" (*_ts) :@@ -72,7 +72,7 @@ static __always_inline notrace long clock_getres_fallback( static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) {long ret;@@ -82,13 +82,13 @@ static __always_inline notrace long clock_gettime_fallback( "call __kernel_vsyscall \n" "mov %%edx, %%ebx \n" : "=a" (ret), "=m" (*_ts) - : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts) + : "0" (__NR_clock_gettime64), [clock] "g" (_clkid), "c" (_ts) : "edx");return ret; } static __always_inline notrace long gettimeofday_fallback( -struct __vdso_timeval *_tv, +struct __kernel_old_timeval *_tv,struct timezone *_tz) {long ret;@@ -105,7 +105,7 @@ static __always_inline notrace long gettimeofday_fallback( static __always_inline notrace long clock_getres_fallback(clockid_t _clkid, -struct __vdso_timespec *_ts) +struct __kernel_timespec *_ts) {long ret;@@ -115,7 +115,7 @@ static __always_inline notrace long clock_getres_fallback( "call __kernel_vsyscall \n" "mov %%edx, %%ebx \n" : "=a" (ret), "=m" (*_ts) - : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts) + : "0" (__NR_clock_getres_time64), [clock] "g" (_clkid), "c" (_ts) : "edx");return ret; }include/vdso/datapage.h +0
-1@@ -9,7 +9,6 @@#include <linux/bits.h> #include <linux/types.h> #include <linux/time.h> -#include <vdso/types.h> #define VDSO_BASES (CLOCK_TAI + 1)#define VDSO_HRES (BIT(CLOCK_REALTIME) | \include/vdso/types.h deleted +0
-39@@ -1,39 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __VDSO_TYPES_H -#define __VDSO_TYPES_H - -#ifdef __KERNEL__ - -#ifndef __ASSEMBLY__ - -#include <linux/types.h> -#include <linux/time.h> - -/* -* The definitions below are required to overcome the limitations -* of time_t on 32 bit architectures, which overflows in 2038. -* The new code should use the replacements based on time64_t and -* timespec64. -* -* The abstraction below will be updated once the migration to -* time64_t is complete. -*/ -#ifdef CONFIG_GENERIC_VDSO_32 -#define __vdso_timespec old_timespec32 -#define __vdso_timeval old_timeval32 -#else -#ifdef ENABLE_COMPAT_VDSO -#define __vdso_timespec old_timespec32 -#define __vdso_timeval old_timeval32 -#else -#define __vdso_timespec __kernel_timespec -#define __vdso_timeval __kernel_old_timeval -#endif /* CONFIG_COMPAT_VDSO */ -#endif /* CONFIG_GENERIC_VDSO_32 */ - - -#endif /* !__ASSEMBLY__ */ - -#endif /* __KERNEL__ */ - -#endif /* __VDSO_TYPES_H */ lib/vdso/gettimeofday.c +35
-13@@ -28,7 +28,7 @@ static notrace int do_hres(const struct vdso_data *vd,clockid_t clk, -struct __vdso_timespec *ts) +struct __kernel_timespec *ts) {const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];u64 cycles, last, sec, ns;@@ -63,7 +63,7 @@ static notrace int do_hres(const struct vdso_data *vd, static notrace void do_coarse(const struct vdso_data *vd,clockid_t clk, -struct __vdso_timespec *ts) +struct __kernel_timespec *ts) {const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];u32 seq;@@ -75,8 +75,8 @@ static notrace void do_coarse(const struct vdso_data *vd, }while (unlikely (vdso_read_retry (vd, seq))); } -static notrace int __cvdso_clock_gettime(clockid_t clock, -struct __vdso_timespec *ts) +static notrace __maybe_unused int +__cvdso_clock_gettime (clockid_t clock,struct __kernel_timespec *ts) {const struct vdso_data *vd =__arch_get_vdso_data ();u32 msk;@@ -100,19 +100,31 @@ static notrace int __cvdso_clock_gettime(clockid_t clock,return clock_gettime_fallback (clock, ts); } -static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv, -struct timezone *tz) +static notrace __maybe_unused int +__cvdso_clock_gettime32 (clockid_t clock,struct old_timespec32 *res) +{ +struct __kernel_timespec ts; +int ret =__cvdso_clock_gettime (clock, &ts); + + res->tv_sec = ts.tv_sec; + res->tv_nsec = ts.tv_nsec; + +return ret; +} + +static notrace __maybe_unused int +__cvdso_gettimeofday (struct __kernel_old_timeval *tv,struct timezone *tz) {const struct vdso_data *vd =__arch_get_vdso_data ();if (likely (tv !=NULL )) { -struct __vdso_timespec ts; +struct __kernel_timespec ts;if (do_hres (vd,CLOCK_REALTIME , &ts))return gettimeofday_fallback (tv, tz); tv->tv_sec = ts.tv_sec; - tv->tv_usec = ts.tv_nsec /NSEC_PER_USEC ; + tv->tv_usec = (u32 )ts.tv_nsec /NSEC_PER_USEC ; }if (unlikely (tz !=NULL )) {@@ -123,8 +135,7 @@ static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,return 0 ; } -#ifdef VDSO_HAS_TIME -static notrace time_t __cvdso_time(time_t *time) +static notrace __maybe_unused time_t __cvdso_time(time_t *time) {const struct vdso_data *vd =__arch_get_vdso_data ();time_t t =READ_ONCE (vd->basetime[CLOCK_REALTIME ].sec);@@ -134,10 +145,9 @@ static notrace time_t __cvdso_time(time_t *time)return t; } -#endif /* VDSO_HAS_TIME */ -static notrace int __cvdso_clock_getres(clockid_t clock, -struct __vdso_timespec *res) +static notrace __maybe_unused +int __cvdso_clock_getres(clockid_t clock, struct __kernel_timespec *res) {u64 sec, ns;u32 msk;@@ -177,3 +187,15 @@ static notrace int __cvdso_clock_getres(clockid_t clock, fallback:return clock_getres_fallback (clock, res); } + +static notrace __maybe_unused int +__cvdso_clock_getres_time32 (clockid_t clock,struct old_timespec32 *res) +{ +struct __kernel_timespec ts; +int ret =__cvdso_clock_getres (clock, &ts); + + res->tv_sec = ts.tv_sec; + res->tv_nsec = ts.tv_nsec; + +return ret; +}
Signature · 2 lines
Regards, Vincenzo
Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
In reply to Vincenzo FrascinoQuoted 9 lines · Vincenzo Frascino
lib/vdso/gettimeofday.c new file +9
-0+static notrace int __cvdso_clock_getres(clockid_t clock, +struct __vdso_timespec *res) +{ +u64 sec, ns; +u32 msk; + +/* Check for negative values or invalid clocks */ +if (unlikely ((u32 ) clock >=MAX_CLOCKS )) + goto fallback;
lib/vdso/gettimeofday.c line 153
Quoted 10 lines
lib/vdso/gettimeofday.c new file +10
-0+ +/* +* Convert the clockid to a bitmask and use it to check which +* clocks are handled in the VDSO directly. +*/ + msk =1U << clock; +if (msk &VDSO_HRES ) { +/* +* Preserves the behaviour of posix_get_hrtimer_res(). +*/
So much for the theory.
lib/vdso/gettimeofday.c line 155
Quoted 2 lines
lib/vdso/gettimeofday.c new file +2
-0+ sec =0 ; + ns =MONOTONIC_RES_NSEC ;
posix_get_hrtimer_res() does:
sec = 0; ns = hrtimer_resolution;
and hrtimer_resolution depends on the enablement of high resolution timers either compile or run time.
So you need to have a copy of hrtimer_resolution in the vdso data and use that.
lib/vdso/gettimeofday.c line 161
Quoted 6 lines
lib/vdso/gettimeofday.c new file +6
-0+ }else if (msk &VDSO_COARSE ) { +/* +* Preserves the behaviour of posix_get_coarse_res(). +*/ + ns =LOW_RES_NSEC ; + sec =__iter_div_u64_rem (ns,NSEC_PER_SEC , &ns);
Do we allow CONFIG_HZ = 1?
Thanks,
tglx
Vincenzo Frascino
maintainerRe: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
Hi Thomas,
thank you for your review.
Quoted 28 lines · Thomas Gleixner
Quoted 9 lines · Vincenzo Frascino
lib/vdso/gettimeofday.c new file +9
-0+static notrace int __cvdso_clock_getres(clockid_t clock, +struct __vdso_timespec *res) +{ +u64 sec, ns; +u32 msk; + +/* Check for negative values or invalid clocks */ +if (unlikely ((u32 ) clock >=MAX_CLOCKS )) + goto fallback;lib/vdso/gettimeofday.c line 153
Quoted 10 lines
lib/vdso/gettimeofday.c new file +10
-0+ +/* +* Convert the clockid to a bitmask and use it to check which +* clocks are handled in the VDSO directly. +*/ + msk =1U << clock; +if (msk &VDSO_HRES ) { +/* +* Preserves the behaviour of posix_get_hrtimer_res(). +*/ So much for the theory.
lib/vdso/gettimeofday.c line 155
Quoted 2 lines
lib/vdso/gettimeofday.c new file +2
-0+ sec =0 ; + ns =MONOTONIC_RES_NSEC ;
posix_get_hrtimer_res()does:
Quoted 9 lines
sec = 0; ns = hrtimer_resolution;and
hrtimer_resolutiondepends on the enablement of high resolution timers either compile or run time.So you need to have a copy of
hrtimer_resolutionin the vdso data and use that.
I agree, MONOTONIC_RES_NSEC can be HIGH_RES_NSEC or LOW_RES_NSEC depending on the HIGH_RES_TIMERS configuration option, but does not cover the run time switch. I will add a copy of hrtimer_resolution in the vdso data in the next iteration of the patches.
I had a look at the other implementations as well, and it seems that all the architectures that currently implement getres() make the same wrong assumption I made. I am going to provide a separate patch set that targets this.
Quoted 9 lines
lib/vdso/gettimeofday.c line 161
Quoted 6 lines
lib/vdso/gettimeofday.c new file +6
-0+ }else if (msk &VDSO_COARSE ) { +/* +* Preserves the behaviour of posix_get_coarse_res(). +*/ + ns =LOW_RES_NSEC ; + sec =__iter_div_u64_rem (ns,NSEC_PER_SEC , &ns);Do we allow
CONFIG_HZ= 1?
I had a closer look at it today and seems that jiffies.h supports CONFIG_HZ=12 as a minimum case. Hence I think it should be safe to remove __iter_div_u64_rem and set sec=0 in this case.
Quoted 4 lines
Thanks,
tglx
Signature · 2 lines
Regards, Vincenzo
Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
In reply to Vincenzo Frascinolib/vdso/gettimeofday.c line 34
Quoted 10 lines · Vincenzo Frascino
lib/vdso/gettimeofday.c new file +10
-0+static notrace int do_hres(const struct vdso_data *vd, +clockid_t clk, +struct __vdso_timespec *ts) +{ +const struct vdso_timestamp *vdso_ts = &vd->basetime[clk]; +u64 cycles, last, sec, ns; +u32 seq, cs_index =CLOCKSOURCE_MONO ; + +if (clk ==CLOCK_MONOTONIC_RAW ) + cs_index =CLOCKSOURCE_RAW ;
Uuurgh. So you create an array with 16 members and then use two. This code is really optimized and now you add not only the pointless array, you also need the extra index plus another conditional. Not to talk about the cache impact which makes things even worse. In the x86 implementation we have:
u32 seq; + 0
int mode; + 4
u64 mask; + 8
u32 mult; + 16
u32 shift; + 20
struct vgtod_ts basetimer[VGTOD_BASES]; + 24Each basetime array member occupies 16 bytes. So
CLOCK_REALTIME + 24 CLOCK_MONOTONIC + 40 .. cacheline boundary .. CLOCK_REALTIME_COARSE + 104 CLOCK_MONOTONIC_COARSE + 120 <- cacheline boundary CLOCK_BOOTTIME + 136 CLOCK_REALTIME_ALARM + 152 CLOCK_BOOTTIME_ALARM + 168
So the most used clocks REALTIME/MONO are in the first cacheline.
So with your scheme the thing becomes
u32 seq; + 0
int mode; + 4
struct cs cs[16] + 8
struct vgtod_ts basetimer[VGTOD_BASES]; + 264and
CLOCK_REALTIME + 264 CLOCK_MONOTONIC + 280
IOW, the most important clocks touch TWO cachelines now which are not even adjacent. No, they are 256 bytes apart, which really sucks for prefetching.
We're surely not going to sacrify the performance which we carefully tuned in that code just to support MONO_RAW. The solution I showed you in the other reply does not have these problems at all.
It's easy enough to benchmark these implementations and without trying I'm pretty sure that you can see the performance drop nicely. Please do so next time and provide the numbers in the changelogs.
Thanks,
tglx
Vincenzo Frascino
maintainerRe: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
Hi Thomas,
Quoted 46 lines · Thomas Gleixner
lib/vdso/gettimeofday.c line 34
Quoted 10 lines · Vincenzo Frascino
lib/vdso/gettimeofday.c new file +10
-0+static notrace int do_hres(const struct vdso_data *vd, +clockid_t clk, +struct __vdso_timespec *ts) +{ +const struct vdso_timestamp *vdso_ts = &vd->basetime[clk]; +u64 cycles, last, sec, ns; +u32 seq, cs_index =CLOCKSOURCE_MONO ; + +if (clk ==CLOCK_MONOTONIC_RAW ) + cs_index =CLOCKSOURCE_RAW ;Uuurgh. So you create an array with 16 members and then use two. This code is really optimized and now you add not only the pointless array, you also need the extra index plus another conditional. Not to talk about the cache impact which makes things even worse. In the x86 implementation we have:
u32 seq; + 0 int mode; + 4 u64 mask; + 8 u32 mult; + 16 u32 shift; + 20 struct vgtod_ts basetimer[VGTOD_BASES]; + 24Each basetime array member occupies 16 bytes. So
CLOCK_REALTIME + 24 CLOCK_MONOTONIC + 40 .. cacheline boundary .. CLOCK_REALTIME_COARSE + 104 CLOCK_MONOTONIC_COARSE + 120 <- cacheline boundary CLOCK_BOOTTIME + 136 CLOCK_REALTIME_ALARM + 152 CLOCK_BOOTTIME_ALARM + 168So the most used clocks REALTIME/MONO are in the first cacheline.
So with your scheme the thing becomes
u32 seq; + 0 int mode; + 4 struct cs cs[16] + 8 struct vgtod_ts basetimer[VGTOD_BASES]; + 264
Quoted 5 lines
and
CLOCK_REALTIME + 264 CLOCK_MONOTONIC + 280
The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and the situation with my scheme should be the following:
u32 seq: + 0 s32 clock_mode; + 4 u64 cycle_last; + 8 struct vdso_cs cs[2]; + 16 struct vdso_ts basetime[VDSO_BASES]; + 48
which I agree makes still things a bit worse.
Assuming L1_CACHE_SHIFT == 6:
CLOCK_REALTIME + 48 ... cache boundary ... CLOCK_MONOTONIC + 64 CLOCK_PROCESS_CPUTIME_ID + 80 CLOCK_THREAD_CPUTIME_ID + 96 CLOCK_MONOTONIC_RAW + 112 ... cache boundary ... CLOCK_REALTIME_COARSE + 128 CLOCK_MONOTONIC_COARSE + 144 CLOCK_BOOTTIME + 160 CLOCK_REALTIME_ALARM + 172 CLOCK_BOOTTIME_ALARM + 188 ...
Quoted 11 lines
IOW, the most important clocks touch TWO cachelines now which are not even adjacent. No, they are 256 bytes apart, which really sucks for prefetching.
We're surely not going to sacrify the performance which we carefully tuned in that code just to support
MONO_RAW. The solution I showed you in the other reply does not have these problems at all.It's easy enough to benchmark these implementations and without trying I'm pretty sure that you can see the performance drop nicely. Please do so next time and provide the numbers in the changelogs.
I did run some benchmarks this morning to quantify the performance impact and seems that using vdsotest[1] the difference in between a stock linux kernel 5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold 5120T), is below 1%. Please find the results below, I will add them as well to the next changelog.
[1] https://github.com/nathanlynch/vdsotest
Quoted 4 lines
Thanks,
tglx
Signature · 115 lines
Regards, Vincenzo 8<----------------- Unified vDSO: ============= clock-gettime-monotonic: syscall: 351 nsec/call clock-gettime-monotonic: libc: 37 nsec/call clock-gettime-monotonic: vdso: 31 nsec/call clock-getres-monotonic: syscall: 271 nsec/call clock-getres-monotonic: libc: 269 nsec/call clock-getres-monotonic: vdso: 9 nsec/call clock-gettime-monotonic-coarse: syscall: 280 nsec/call clock-gettime-monotonic-coarse: libc: 22 nsec/call clock-gettime-monotonic-coarse: vdso: 11 nsec/call clock-getres-monotonic-coarse: syscall: 274 nsec/call clock-getres-monotonic-coarse: libc: 276 nsec/call clock-getres-monotonic-coarse: vdso: 10 nsec/call clock-gettime-monotonic-raw: syscall: 337 nsec/call clock-gettime-monotonic-raw: libc: 38 nsec/call clock-gettime-monotonic-raw: vdso: 32 nsec/call clock-getres-monotonic-raw: syscall: 284 nsec/call clock-getres-monotonic-raw: libc: 271 nsec/call clock-getres-monotonic-raw: vdso: 9 nsec/call clock-gettime-tai: syscall: 332 nsec/call clock-gettime-tai: libc: 37 nsec/call clock-gettime-tai: vdso: 31 nsec/call clock-getres-tai: syscall: 273 nsec/call clock-getres-tai: libc: 281 nsec/call clock-getres-tai: vdso: 10 nsec/call clock-gettime-boottime: syscall: 338 nsec/call clock-gettime-boottime: libc: 37 nsec/call clock-gettime-boottime: vdso: 32 nsec/call clock-getres-boottime: syscall: 283 nsec/call clock-getres-boottime: libc: 278 nsec/call clock-getres-boottime: vdso: 9 nsec/call clock-gettime-realtime: syscall: 338 nsec/call clock-gettime-realtime: libc: 39 nsec/call clock-gettime-realtime: vdso: 32 nsec/call clock-getres-realtime: syscall: 281 nsec/call clock-getres-realtime: libc: 277 nsec/call clock-getres-realtime: vdso: 10 nsec/call clock-gettime-realtime-coarse: syscall: 286 nsec/call clock-gettime-realtime-coarse: libc: 21 nsec/call clock-gettime-realtime-coarse: vdso: 12 nsec/call clock-getres-realtime-coarse: syscall: 285 nsec/call clock-getres-realtime-coarse: libc: 283 nsec/call clock-getres-realtime-coarse: vdso: 11 nsec/call getcpu: syscall: 234 nsec/call getcpu: libc: 31 nsec/call getcpu: vdso: 20 nsec/call gettimeofday: syscall: 293 nsec/call gettimeofday: libc: 32 nsec/call gettimeofday: vdso: 31 nsec/call Stock Kernel: ============= clock-gettime-monotonic: syscall: 349 nsec/call clock-gettime-monotonic: libc: 37 nsec/call clock-gettime-monotonic: vdso: 28 nsec/call clock-getres-monotonic: syscall: 296 nsec/call clock-getres-monotonic: libc: 295 nsec/call clock-getres-monotonic: vdso: not tested Note: vDSO version of clock_getres not found clock-gettime-monotonic-coarse: syscall: 296 nsec/call clock-gettime-monotonic-coarse: libc: 21 nsec/call clock-gettime-monotonic-coarse: vdso: 11 nsec/call clock-getres-monotonic-coarse: syscall: 287 nsec/call clock-getres-monotonic-coarse: libc: 288 nsec/call clock-getres-monotonic-coarse: vdso: not tested Note: vDSO version of clock_getres not found clock-gettime-monotonic-raw: syscall: 353 nsec/call clock-gettime-monotonic-raw: libc: 360 nsec/call clock-gettime-monotonic-raw: vdso: 352 nsec/call clock-getres-monotonic-raw: syscall: 282 nsec/call clock-getres-monotonic-raw: libc: 286 nsec/call clock-getres-monotonic-raw: vdso: not tested Note: vDSO version of clock_getres not found clock-gettime-tai: syscall: 351 nsec/call clock-gettime-tai: libc: 364 nsec/call clock-gettime-tai: vdso: 365 nsec/call clock-getres-tai: syscall: 287 nsec/call clock-getres-tai: libc: 287 nsec/call clock-getres-tai: vdso: not tested Note: vDSO version of clock_getres not found clock-gettime-boottime: syscall: 347 nsec/call clock-gettime-boottime: libc: 364 nsec/call clock-gettime-boottime: vdso: 355 nsec/call clock-getres-boottime: syscall: 287 nsec/call clock-getres-boottime: libc: 287 nsec/call clock-getres-boottime: vdso: not tested Note: vDSO version of clock_getres not found clock-gettime-realtime: syscall: 346 nsec/call clock-gettime-realtime: libc: 36 nsec/call clock-gettime-realtime: vdso: 29 nsec/call clock-getres-realtime: syscall: 285 nsec/call clock-getres-realtime: libc: 287 nsec/call clock-getres-realtime: vdso: not tested Note: vDSO version of clock_getres not found clock-gettime-realtime-coarse: syscall: 296 nsec/call clock-gettime-realtime-coarse: libc: 20 nsec/call clock-gettime-realtime-coarse: vdso: 11 nsec/call clock-getres-realtime-coarse: syscall: 301 nsec/call clock-getres-realtime-coarse: libc: 297 nsec/call clock-getres-realtime-coarse: vdso: not tested Note: vDSO version of clock_getres not found getcpu: syscall: 255 nsec/call getcpu: libc: 32 nsec/call getcpu: vdso: 21 nsec/call gettimeofday: syscall: 339 nsec/call gettimeofday: libc: 31 nsec/call gettimeofday: vdso: 30 nsec/call
Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
Vincenzo,
Quoted 3 lines · Vincenzo Frascino
The clocksource array has two elements (
CLOCKSOURCE_RAW,CLOCKSOURCE_MONO) and the situation with my scheme should be the following:
Oops. I misread the patch, but still...
Quoted 13 lines
u32 seq: + 0 s32 clock_mode; + 4 u64 cycle_last; + 8 struct vdso_cs cs[2]; + 16 struct vdso_ts basetime[VDSO_BASES]; + 48which I agree makes still things a bit worse.
Quoted 4 lines
It's easy enough to benchmark these implementations and without trying I'm pretty sure that you can see the performance drop nicely. Please do so next time and provide the numbers in the changelogs.
Quoted 5 lines
I did run some benchmarks this morning to quantify the performance impact and seems that using vdsotest[1] the difference in between a stock linux kernel 5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold 5120T), is below 1%. Please find the results below, I will add them as well to the next changelog.
I have some doubts about 1%.
NEW STOCK
clock-gettime-monotonic: vdso: 31 28 ~ 10% slower clock-gettime-realtime: vdso: 32 29 ~ 10% slower
Thanks,
tglx
Vincenzo Frascino
maintainerRe: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
Quoted 28 lines · Thomas Gleixner
Vincenzo,
Quoted 3 lines · Vincenzo Frascino
The clocksource array has two elements (
CLOCKSOURCE_RAW,CLOCKSOURCE_MONO) and the situation with my scheme should be the following:Oops. I misread the patch, but still...
Quoted 13 lines
u32 seq: + 0 s32 clock_mode; + 4 u64 cycle_last; + 8 struct vdso_cs cs[2]; + 16 struct vdso_ts basetime[VDSO_BASES]; + 48which I agree makes still things a bit worse.
Quoted 4 lines
It's easy enough to benchmark these implementations and without trying I'm pretty sure that you can see the performance drop nicely. Please do so next time and provide the numbers in the changelogs.
Quoted 5 lines
I did run some benchmarks this morning to quantify the performance impact and seems that using vdsotest[1] the difference in between a stock linux kernel 5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold 5120T), is below 1%. Please find the results below, I will add them as well to the next changelog.
Quoted 5 lines
I have some doubts about 1%.
NEW STOCKclock-gettime-monotonic: vdso: 31 28 ~ 10% slower clock-gettime-realtime: vdso: 32 29 ~ 10% slower
Sorry, there was an error in my script that I used to extract percentages. I agree! Luckily I shared the numbers. Thanks Thomas.
Quoted 4 lines
Thanks,
tglx
Signature · 2 lines
Regards, Vincenzo
Vincenzo Frascino
maintainerPATCH v5 3/23 arm64: Build vDSO with -ffixed-x18
In reply to Vincenzo FrascinoFrom: Peter Collingbourne <[email protected]>
The vDSO needs to be build with x18 reserved in order to accommodate userspace platform ABIs built on top of Linux that use the register to carry inter-procedural state, as provided for by the AAPCS. An example of such a platform ABI is the one that will be used by an upcoming version of Android.
Although this change is currently a no-op due to the fact that the vDSO is currently implemented in pure assembly on arm64, it is necessary in order to prepare for another change [1] that will add C code to the vDSO.
[1] https://patchwork.kernel.org/patch/10044501/
--- arch/arm64/kernel/vdso/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kernel/vdso/Makefile +1 -1
@@ -12,7 +12,7 @@ obj-vdso := gettimeofday.o note.o sigreturn.o targets := $(obj-vdso) vdso.so vdso.so.dbg obj-vdso := $(addprefix $(obj)/, $(obj-vdso)) -ccflags-y := -shared -fno-common -fno-builtin +ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \ $(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 4/23 arm64: Substitute gettimeofday with C implementation
In reply to Vincenzo FrascinoTo take advantage of the commonly defined vdso interface for gettimeofday the architectural code requires an adaptation.
Re-implement the gettimeofday vdso in C in order to use lib/vdso.
With the new implementation arm64 gains support for CLOCK_BOOTTIME and CLOCK_TAI.
--- arch/arm64/Kconfig | 2 + arch/arm64/include/asm/vdso/gettimeofday.h | 85 ++++++ arch/arm64/include/asm/vdso_datapage.h | 47 --- arch/arm64/kernel/asm-offsets.c | 39 ++- arch/arm64/kernel/vdso.c | 102 +++++-- arch/arm64/kernel/vdso/Makefile | 28 +- arch/arm64/kernel/vdso/gettimeofday.S | 328 --------------------- arch/arm64/kernel/vdso/vgettimeofday.c | 28 ++ 8 files changed, 237 insertions(+), 422 deletions(-) create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h delete mode 100644 arch/arm64/include/asm/vdso_datapage.h delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
arch/arm64/Kconfig +2 -0
@@ -100,6 +100,7 @@ config ARM64 select GENERIC_STRNCPY_FROM_USER select GENERIC_STRNLEN_USER select GENERIC_TIME_VSYSCALL + select GENERIC_GETTIMEOFDAY select HANDLE_DOMAIN_IRQ select HARDIRQS_SW_RESEND select HAVE_PCI
@@ -154,6 +155,7 @@ config ARM64 select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES select HAVE_KRETPROBES + select HAVE_GENERIC_VDSO select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING
arch/arm64/include/asm/vdso/gettimeofday.h new file +85 -0
85 lines not on this page · raw message
arch/arm64/include/asm/vdso_datapage.h deleted +0 -47
47 lines not on this page · raw message
arch/arm64/kernel/asm-offsets.c +24 -15
@@ -25,13 +25,13 @@#include <linux/kvm_host.h> #include <linux/preempt.h> #include <linux/suspend.h> +#include <vdso/datapage.h> #include <asm/cpufeature.h> #include <asm/fixmap.h> #include <asm/thread_info.h> #include <asm/memory.h> #include <asm/smp_plat.h> #include <asm/suspend.h> -#include <asm/vdso_datapage.h> #include <linux/kbuild.h> #include <linux/arm-smccc.h>
@@ -108,22 +108,31 @@ int main(void)DEFINE (CLOCK_COARSE_RES ,LOW_RES_NSEC );DEFINE (NSEC_PER_SEC ,NSEC_PER_SEC );BLANK (); -DEFINE (VDSO_CS_CYCLE_LAST , offsetof(struct vdso_data , cs_cycle_last)); -DEFINE (VDSO_RAW_TIME_SEC , offsetof(struct vdso_data , raw_time_sec)); -DEFINE (VDSO_RAW_TIME_NSEC , offsetof(struct vdso_data , raw_time_nsec)); -DEFINE (VDSO_XTIME_CLK_SEC , offsetof(struct vdso_data , xtime_clock_sec)); -DEFINE (VDSO_XTIME_CLK_NSEC , offsetof(struct vdso_data , xtime_clock_nsec)); -DEFINE (VDSO_XTIME_CRS_SEC , offsetof(struct vdso_data , xtime_coarse_sec)); -DEFINE (VDSO_XTIME_CRS_NSEC , offsetof(struct vdso_data , xtime_coarse_nsec)); -DEFINE (VDSO_WTM_CLK_SEC , offsetof(struct vdso_data , wtm_clock_sec)); -DEFINE (VDSO_WTM_CLK_NSEC , offsetof(struct vdso_data , wtm_clock_nsec)); -DEFINE (VDSO_TB_SEQ_COUNT , offsetof(struct vdso_data , tb_seq_count)); -DEFINE (VDSO_CS_MONO_MULT , offsetof(struct vdso_data , cs_mono_mult)); -DEFINE (VDSO_CS_RAW_MULT , offsetof(struct vdso_data , cs_raw_mult)); -DEFINE (VDSO_CS_SHIFT , offsetof(struct vdso_data , cs_shift)); +DEFINE (VDSO_SEQ , offsetof(struct vdso_data , seq)); +DEFINE (VDSO_CLK_MODE , offsetof(struct vdso_data , clock_mode)); +DEFINE (VDSO_CYCLE_LAST , offsetof(struct vdso_data , cycle_last)); + DEFINE(VDSO_MONO_MASK, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].mask)); + DEFINE(VDSO_MONO_MULT, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].mult)); + DEFINE(VDSO_MONO_SHIFT, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].shift)); + DEFINE(VDSO_RAW_MASK, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].mask)); + DEFINE(VDSO_RAW_MULT, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].mult)); + DEFINE(VDSO_RAW_SHIFT, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].shift)); + DEFINE(VDSO_REALTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].sec)); + DEFINE(VDSO_REALTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].nsec)); + DEFINE(VDSO_MONO_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].sec)); + DEFINE(VDSO_MONO_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].nsec)); + DEFINE(VDSO_MONO_RAW_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].sec)); + DEFINE(VDSO_MONO_RAW_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].nsec)); + DEFINE(VDSO_BOOTTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].sec)); + DEFINE(VDSO_BOOTTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].nsec)); + DEFINE(VDSO_TAI_SEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].sec)); + DEFINE(VDSO_TAI_NSEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].nsec)); + DEFINE(VDSO_RT_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].sec)); + DEFINE(VDSO_RT_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].nsec)); + DEFINE(VDSO_MONO_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].sec)); + DEFINE(VDSO_MONO_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].nsec));DEFINE (VDSO_TZ_MINWEST , offsetof(struct vdso_data , tz_minuteswest));DEFINE (VDSO_TZ_DSTTIME , offsetof(struct vdso_data , tz_dsttime)); -DEFINE (VDSO_USE_SYSCALL , offsetof(struct vdso_data , use_syscall));BLANK ();DEFINE (TVAL_TV_SEC , offsetof(struct timeval , tv_sec));DEFINE (TVAL_TV_USEC , offsetof(struct timeval , tv_usec));
arch/arm64/kernel/vdso.c +79 -23
128 lines not on this page · raw message
arch/arm64/kernel/vdso/Makefile +19 -9
@@ -6,16 +6,31 @@ # Heavily based on the vDSO Makefiles for other archs. # -obj-vdso := gettimeofday.o note.o sigreturn.o +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before +# the inclusion of generic Makefile. +ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64 +include $(srctree)/lib/vdso/Makefile + +obj-vdso := vgettimeofday.o note.o sigreturn.o # Build rules targets := $(obj-vdso) vdso.so vdso.so.dbg obj-vdso := $(addprefix $(obj)/, $(obj-vdso)) -ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 +ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector +ccflags-y += -DDISABLE_BRANCH_PROFILING ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \ $(call cc-ldoption, -Wl$(comma)--hash-style=sysv) +VDSO_LDFLAGS := -Bsymbolic + +CFLAGS_REMOVE_vgettimeofday.o = -pg -Os +ifeq ($(c-gettimeofday-y),) +CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny +else +CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y) +endif + # Disable gcov profiling for VDSO code GCOV_PROFILE := n
@@ -33,6 +48,7 @@ $(obj)/vdso.o : $(obj)/vdso.so # Link rule for the .so file, .lds has to be first $(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso) $(call if_changed,vdsold) + $(call if_changed,vdso_check) # Strip rule for the .so file $(obj)/%.so: OBJCOPYFLAGS := -S
@@ -49,15 +65,9 @@ endef include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE $(call if_changed,vdsosym) -# Assembly rules for the .S files -$(obj-vdso): %.o: %.S FORCE - $(call if_changed_dep,vdsoas) - # Actual build commands quiet_cmd_vdsold = VDSOL $@ - cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@ -quiet_cmd_vdsoas = VDSOA $@ - cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $< + cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) -Wl,-n -Wl,-T $^ -o $@ # Install commands for the unstripped file quiet_cmd_vdso_install = INSTALL $@
arch/arm64/kernel/vdso/gettimeofday.S deleted +0 -328
328 lines not on this page · raw message
arch/arm64/kernel/vdso/vgettimeofday.c new file +28 -0
@@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* ARM64 userspace implementations of gettimeofday() and similar. +* +* Copyright (C) 2018 ARM Limited +* +*/ +#include <linux/time.h> +#include <linux/types.h> + +notrace int __kernel_clock_gettime(clockid_t clock, +struct __vdso_timespec *ts) +{ +return __cvdso_clock_gettime (clock, ts); +} + +notrace int __kernel_gettimeofday(struct __vdso_timeval *tv, +struct timezone *tz) +{ +return __cvdso_gettimeofday (tv, tz); +} + +notrace int __kernel_clock_getres(clockid_t clock_id, +struct __vdso_timespec *res) +{ +return __cvdso_clock_getres (clock_id, res); +} +
Signature · 1 line
2.20.1
Mark Rutland
reviewerRe: [PATCH v5 04/23] arm64: Substitute gettimeofday with C implementation
Quoted 19 lines · Vincenzo Frascino
arch/arm64/kernel/vdso/Makefile +6
-1@@ -6,16 +6,31 @@ # Heavily based on the vDSO Makefiles for other archs. # -obj-vdso := gettimeofday.o note.o sigreturn.o +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before +# the inclusion of generic Makefile. +ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64 +include $(srctree)/lib/vdso/Makefile + +obj-vdso := vgettimeofday.o note.o sigreturn.o # Build rules targets := $(obj-vdso) vdso.so vdso.so.dbg obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
arch/arm64/kernel/vdso/Makefile line 27
Quoted 10 lines
arch/arm64/kernel/vdso/Makefile +5
-1-ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 +ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector +ccflags-y += -DDISABLE_BRANCH_PROFILING ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \ $(call cc-ldoption, -Wl$(comma)--hash-style=sysv) +VDSO_LDFLAGS := -Bsymbolic + +CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
Please use CC_FLAGS_FTRACE rather than '-pg'
What said, wouldn't it be better to come up with the VDSO cflags entirely separately from what the main kernel uses?
That way we're unlikely to have any surprises in future with instrumentation etc.
I strongly suspect you need to disable more things here, e.g. UBSAN, KASAN, KCOV. Have you tried compiling a kernel with those features enabled?
VDSO files probably also need to be marked as OBJECT_FILES_NON_STANDARD.
Thanks,
Mark.
Vincenzo Frascino
maintainerPATCH v5 5/23 arm64: compat: Alloc separate pages for vectors and sigpage
In reply to Vincenzo FrascinoIn the current implementation AArch32 installs a special page called "[vectors]" that contains sigreturn trampolines and kuser helpers, and this is done at fixed address specified by the kuser helpers ABI.
Having sigreturn trampolines and kuser helpers in the same page, makes difficult to maintain compatibility with arm because it makes not possible to disable kuser helpers.
Address the problem creating separate pages for vectors and sigpage in a similar fashion to what happens today on arm.
Change as well the meaning of mm->context.vdso for AArch32 compat since it now points to sigpage and not to vectors anymore in order to make simpler the implementation of the signal handling (the address of sigpage is randomized).
--- arch/arm64/include/asm/elf.h | 6 +- arch/arm64/include/asm/processor.h | 4 +- arch/arm64/include/asm/signal32.h | 2 - arch/arm64/kernel/signal32.c | 5 +- arch/arm64/kernel/vdso.c | 112 ++++++++++++++++++++++------- 5 files changed, 93 insertions(+), 36 deletions(-)
arch/arm64/include/asm/elf.h +3 -3
@@ -214,10 +214,10 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];set_thread_flag (TIF_32BIT ); \ })#define COMPAT_ARCH_DLINFO -extern int aarch32_setup_vectors_page (struct linux_binprm *bprm, -int uses_interp); +extern int aarch32_setup_additional_pages (struct linux_binprm *bprm, +int uses_interp);#define compat_arch_setup_additional_pages \ - aarch32_setup_vectors_page + aarch32_setup_additional_pages#endif /* CONFIG_COMPAT */
arch/arm64/include/asm/processor.h +2 -2
@@ -78,9 +78,9 @@#endif /* CONFIG_ARM64_FORCE_52BIT */ #ifdef CONFIG_COMPAT -#define AARCH32_VECTORS_BASE 0xffff0000 +#define AARCH32_KUSER_BASE 0xffff0000#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \ - AARCH32_VECTORS_BASE : STACK_TOP_MAX) + AARCH32_KUSER_BASE : STACK_TOP_MAX)#else #define STACK_TOP STACK_TOP_MAX#endif /* CONFIG_COMPAT */
arch/arm64/include/asm/signal32.h +0 -2
@@ -20,8 +20,6 @@#ifdef CONFIG_COMPAT #include <linux/compat.h> -#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500 -int compat_setup_frame (int usig,struct ksignal *ksig,sigset_t *set,struct pt_regs *regs); int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
arch/arm64/kernel/signal32.c +3 -2
@@ -379,6 +379,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,compat_ulong_t retcode;compat_ulong_t spsr = regs->pstate & ~(PSR_f |PSR_AA32_E_BIT );int thumb; +void *sigreturn_base;/* Check if the handler is written for ARM or Thumb */ thumb = handler &1 ;
@@ -399,12 +400,12 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka, }else {/* Set up sigreturn pointer */ unsigned int idx = thumb <<1 ; + sigreturn_base = current->mm->context.vdso;if (ka->sa.sa_flags &SA_SIGINFO ) idx +=3 ; - retcode =AARCH32_VECTORS_BASE + -AARCH32_KERN_SIGRET_CODE_OFFSET + + retcode =ptr_to_compat (sigreturn_base) + (idx <<2 ) + thumb; }
arch/arm64/kernel/vdso.c +85 -27
151 lines not on this page · raw message
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 6/23 arm64: compat: Split kuser32
In reply to Vincenzo FrascinoTo make it possible to disable kuser helpers in aarch32 we need to divide the kuser and the sigreturn functionalities.
Split the current version of kuser32 in kuser32 (for kuser helpers) and sigreturn32 (for sigreturn helpers).
--- arch/arm64/kernel/Makefile | 2 +- arch/arm64/kernel/kuser32.S | 58 ++------------------------------- arch/arm64/kernel/sigreturn32.S | 46 ++++++++++++++++++++++++++ 3 files changed, 49 insertions(+), 57 deletions(-) create mode 100644 arch/arm64/kernel/sigreturn32.S
arch/arm64/kernel/Makefile +1 -1
@@ -28,7 +28,7 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE $(call if_changed,objcopy) obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \ - sys_compat.o + sigreturn32.o sys_compat.o obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o obj-$(CONFIG_MODULES) += module.o obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
arch/arm64/kernel/kuser32.S +2 -56
68 lines not on this page · raw message
arch/arm64/kernel/sigreturn32.S new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 7/23 arm64: compat: Refactor aarch32_alloc_vdso_pages()
In reply to Vincenzo Frascinoaarch32_alloc_vdso_pages() needs to the refactored to make it easier to disable kuser helpers.
Divide the function in aarch32_alloc_kuser_vdso_page() and aarch32_alloc_sigreturn_vdso_page().
--- arch/arm64/kernel/vdso.c | 49 ++++++++++++++++++++++++++-------------- 1 file changed, 32 insertions(+), 17 deletions(-)
arch/arm64/kernel/vdso.c +32 -17
@@ -75,40 +75,55 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = { }, }; -static int __init aarch32_alloc_vdso_pages (void ) +static int aarch32_alloc_kuser_vdso_page (void ) {extern char __kuser_helper_start[], __kuser_helper_end[]; -extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[]; -int kuser_sz = __kuser_helper_end - __kuser_helper_start; -int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start; -unsigned long vdso_pages[2 ]; - - vdso_pages[0 ] =get_zeroed_page (GFP_ATOMIC ); -if (!vdso_pages[0 ]) -return -ENOMEM ; +unsigned long vdso_page; - vdso_pages[1 ] =get_zeroed_page (GFP_ATOMIC ); -if (!vdso_pages[1 ]) + vdso_page =get_zeroed_page (GFP_ATOMIC ); +if (!vdso_page)return -ENOMEM ;/* kuser helpers */ -memcpy ((void *)(vdso_pages[0 ] +0x1000 - kuser_sz), +memcpy ((void *)(vdso_page +0x1000 - kuser_sz), __kuser_helper_start, kuser_sz); +flush_icache_range (vdso_page, vdso_page +PAGE_SIZE ); + + aarch32_vdso_pages[0 ] =virt_to_page (vdso_page); + +return 0 ; +} + +static int aarch32_alloc_sigreturn_vdso_page (void ) +{ +extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[]; +int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start; +unsigned long vdso_page; + + vdso_page =get_zeroed_page (GFP_ATOMIC ); +if (!vdso_page) +return -ENOMEM ; +/* sigreturn code */ -memcpy ((void *)vdso_pages[1 ], +memcpy ((void *)vdso_page, __aarch32_sigret_code_start, sigret_sz); -flush_icache_range (vdso_pages[0 ], vdso_pages[0 ] +PAGE_SIZE ); -flush_icache_range (vdso_pages[1 ], vdso_pages[1 ] +PAGE_SIZE ); +flush_icache_range (vdso_page, vdso_page +PAGE_SIZE ); - aarch32_vdso_pages[0 ] =virt_to_page (vdso_pages[0 ]); - aarch32_vdso_pages[1 ] =virt_to_page (vdso_pages[1 ]); + aarch32_vdso_pages[1 ] =virt_to_page (vdso_page);return 0 ; + +} + +static int __init aarch32_alloc_vdso_pages (void ) +{ +return aarch32_alloc_kuser_vdso_page () & +aarch32_alloc_sigreturn_vdso_page (); }arch_initcall (aarch32_alloc_vdso_pages);
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 8/23 arm64: compat: Add KUSER_HELPERS config option
In reply to Vincenzo FrascinoWhen kuser helpers are enabled the kernel maps the relative code at a fixed address (0xffff0000). Making configurable the option to disable them means that the kernel can remove this mapping and any access to this memory area results in a sigfault.
Add a KUSER_HELPERS config option that can be used to disable the mapping when it is turned off.
This option can be turned off if and only if the applications are designed specifically for the platform and they do not make use of the kuser helpers code.
--- arch/arm64/Kconfig | 21 +++++++++++++++++++++ arch/arm64/kernel/Makefile | 3 ++- arch/arm64/kernel/kuser32.S | 7 +++---- arch/arm64/kernel/vdso.c | 15 +++++++++++++++ 4 files changed, 41 insertions(+), 5 deletions(-)
arch/arm64/Kconfig +21 -0
@@ -1465,6 +1465,27 @@ config COMPAT If you want to execute 32-bit userspace applications, say Y. +config KUSER_HELPERS + bool "Enable kuser helpers page for compatibility with 32 bit applications." + depends on COMPAT + default y + help + Enables kuser helpers to be mapped in a special purpose page at a fixed + address to maintain independence from the type of CPU present in the SoC. + This feature is provided for compatibility reasons in fact allows 32 bit + applications compliant with ARMv4 up to ARMv8 to run without any + modification. + + Warning: Being always mapped at a fixed address makes it easier to create + exploits based on ROP type of attacks. + + As a consequence of this, this feature is made configurable but be aware that + it can be turned off if and only if the binaries and the libraries running on + a specific platform are designed to do not make use of these helpers, otherwise + should be left on. + + See Documentation/arm/kernel_user_helpers.txt for details. + config SYSVIPC_COMPAT def_bool y depends on COMPAT && SYSVIPC
arch/arm64/kernel/Makefile +2 -1
@@ -27,8 +27,9 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_ $(obj)/%.stub.o: $(obj)/%.o FORCE $(call if_changed,objcopy) -obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \ +obj-$(CONFIG_COMPAT) += sys32.o signal32.o \ sigreturn32.o sys_compat.o +obj-$(CONFIG_KUSER_HELPERS) += kuser32.o obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o obj-$(CONFIG_MODULES) += module.o obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
arch/arm64/kernel/kuser32.S +3 -4
@@ -5,10 +5,9 @@ * Copyright (C) 2005-2011 Nicolas Pitre <[email protected]> * Copyright (C) 2012-2018 ARM Ltd. * - * Each segment is 32-byte aligned and will be moved to the top of the high - * vector page. New segments (if ever needed) must be added in front of - * existing ones. This mechanism should be used only for things that are - * really small and justified, and not be abused freely. + * The kuser helpers below are mapped at a fixed address by + * aarch32_setup_additional_pages() ad are provided for compatibility + * reasons with 32 bit (aarch32) applications that need them. * * See Documentation/arm/kernel_user_helpers.txt for formal definitions. */
arch/arm64/kernel/vdso.c +15 -0
@@ -75,6 +75,7 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = { }, }; +#ifdef CONFIG_KUSER_HELPERS static int aarch32_alloc_kuser_vdso_page (void ) {extern char __kuser_helper_start[], __kuser_helper_end[];
@@ -96,6 +97,12 @@ static int aarch32_alloc_kuser_vdso_page(void) return 0; } +#else +static int aarch32_alloc_kuser_vdso_page (void ) +{ +return 0 ; +} +#endif /* CONFIG_KUSER_HELPER */ static int aarch32_alloc_sigreturn_vdso_page (void ) {
@@ -127,6 +134,7 @@ static int __init aarch32_alloc_vdso_pages(void) }arch_initcall (aarch32_alloc_vdso_pages); +#ifdef CONFIG_KUSER_HELPERS static int aarch32_kuser_helpers_setup (struct mm_struct *mm) {void *ret;
@@ -139,6 +147,13 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm) return PTR_ERR_OR_ZERO(ret); } +#else +static int aarch32_kuser_helpers_setup (struct mm_struct *mm) +{ +/* kuser helpers not enabled */ +return 0 ; +} +#endif /* CONFIG_KUSER_HELPERS */ static int aarch32_sigreturn_setup (struct mm_struct *mm) {
Signature · 1 line
2.20.1
Mark Rutland
reviewerRe: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
Quoted 8 lines · Vincenzo Frascino
When kuser helpers are enabled the kernel maps the relative code at a fixed address (
0xffff0000). Making configurable the option to disable them means that the kernel can remove this mapping and any access to this memory area results in a sigfault.Add a
KUSER_HELPERSconfig option that can be used to disable the mapping when it is turned off.
Quoted 7 lines
This option can be turned off if and only if the applications are designed specifically for the platform and they do not make use of the kuser helpers code.
Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Signed-off-by Vincenzo Frascino <[email protected]>
IIUC, the KUSER bits aren't striclty dependent on the rest of the vDSO changes.
I think the KUSER bits should be split into a preparatory series, so that they can be merged soon, rather than tying them artificially to the rest of the vDSO bits (which I expect will require significantly more review).
Quoted 25 lines
--- arch/arm64/Kconfig | 21 +++++++++++++++++++++ arch/arm64/kernel/Makefile | 3 ++- arch/arm64/kernel/kuser32.S | 7 +++---- arch/arm64/kernel/vdso.c | 15 +++++++++++++++ 4 files changed, 41 insertions(+), 5 deletions(-)arch/arm64/Kconfig +10
-0@@ -1465,6 +1465,27 @@ config COMPAT If you want to execute 32-bit userspace applications, say Y. +config KUSER_HELPERS + bool "Enable kuser helpers page for compatibility with 32 bit applications." + depends on COMPAT + default y + help + Enables kuser helpers to be mapped in a special purpose page at a fixed + address to maintain independence from the type of CPU present in the SoC. + This feature is provided for compatibility reasons in fact allows 32 bit + applications compliant with ARMv4 up to ARMv8 to run without any + modification.
arch/arm64/Kconfig config COMPAT · line 1487
Quoted 10 lines
arch/arm64/Kconfig +10
-0+ + Warning: Being always mapped at a fixed address makes it easier to create + exploits based on ROP type of attacks. + + As a consequence of this, this feature is made configurable but be aware that + it can be turned off if and only if the binaries and the libraries running on + a specific platform are designed to do not make use of these helpers, otherwise + should be left on. + + See Documentation/arm/kernel_user_helpers.txt for details.
This is quite divergent from the text for the 32-bit arm KUSER_HELPERS.
Could we please reuse that existing text as far as possible, with minimal changes?
e.g.
config KUSER_HELPERS
bool "Enable kuser helpers for compat tasks" depends on COMPAT
default y
help
Warning: disabling this option may break user programs.
Provide kuser helpers to compat tasks. The kernel provides helper code to userspace in read only form at a fixed location to allow userspace to be independent of the CPU type fitted to the system. This permits binaries to be run on ARMv4 through to ARMv8 without modification.
See Documentation/arm/kernel_user_helpers.txt for details.
However, the fixed address nature of these helpers can be used by ROP (return orientated programming) authors when creating exploits.
If all of the binaries and libraries which run on your platform are built specifically for your platform, and make no use of these helpers, then you can turn this option off to hinder such exploits. However, in that case, if a binary or library relying on those helpers is run, it will not function correctly.
Say N here only if you are absolutely certain that you do not need these helpers; otherwise, the safe option is to say Y.
I believe that Will wanted to default the kuser helpers off, as applications using them need the instruction emulation which is also disabled by default.
I'm also not sure which binaries we expect to work on arm64, so I'm not sure if the ARMv4 reference is correct.
Thanks,
Mark.
Russell King - ARM Linux admin
maintainerRe: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
Quoted 5 lines · Mark Rutland
I believe that Will wanted to default the kuser helpers off, as applications using them need the instruction emulation which is also disabled by default.
I'm also not sure which binaries we expect to work on arm64, so I'm not sure if the ARMv4 reference is correct.
Do you expect ARMv4 - even ARMv4 OABI binaries to fail on ARM64?
(I've run my OABI ARMv4 distro under KVM on Macchiatobin when I've needed to rebuild a RPM without problems - using its own gcc. No observable issues.)
Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up According to speedtest.net: 11.9Mbps down 500kbps up
Vincenzo Frascino
maintainerRe: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
In reply to Mark RutlandHi Mark,
thank you for your review.
Quoted 17 lines · Mark Rutland
Quoted 8 lines · Vincenzo Frascino
When kuser helpers are enabled the kernel maps the relative code at a fixed address (
0xffff0000). Making configurable the option to disable them means that the kernel can remove this mapping and any access to this memory area results in a sigfault.Add a
KUSER_HELPERSconfig option that can be used to disable the mapping when it is turned off.Quoted 7 lines
This option can be turned off if and only if the applications are designed specifically for the platform and they do not make use of the kuser helpers code.
Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Signed-off-by Vincenzo Frascino <[email protected]>
Quoted 7 lines
IIUC, the KUSER bits aren't striclty dependent on the rest of the vDSO changes.
I think the KUSER bits should be split into a preparatory series, so that they can be merged soon, rather than tying them artificially to the rest of the vDSO bits (which I expect will require significantly more review).
This is a good point. I will extract them and post as a separate patch set. In doing so I will update the config description below.
Signature · 2 lines
Regards, Vincenzo
Quoted 80 lines
Quoted 25 lines
--- arch/arm64/Kconfig | 21 +++++++++++++++++++++ arch/arm64/kernel/Makefile | 3 ++- arch/arm64/kernel/kuser32.S | 7 +++---- arch/arm64/kernel/vdso.c | 15 +++++++++++++++ 4 files changed, 41 insertions(+), 5 deletions(-)arch/arm64/Kconfig +10
-0@@ -1465,6 +1465,27 @@ config COMPAT If you want to execute 32-bit userspace applications, say Y. +config KUSER_HELPERS + bool "Enable kuser helpers page for compatibility with 32 bit applications." + depends on COMPAT + default y + help + Enables kuser helpers to be mapped in a special purpose page at a fixed + address to maintain independence from the type of CPU present in the SoC. + This feature is provided for compatibility reasons in fact allows 32 bit + applications compliant with ARMv4 up to ARMv8 to run without any + modification.arch/arm64/Kconfig config COMPAT · line 1487
Quoted 10 lines
arch/arm64/Kconfig +10
-0+ + Warning: Being always mapped at a fixed address makes it easier to create + exploits based on ROP type of attacks. + + As a consequence of this, this feature is made configurable but be aware that + it can be turned off if and only if the binaries and the libraries running on + a specific platform are designed to do not make use of these helpers, otherwise + should be left on. + + See Documentation/arm/kernel_user_helpers.txt for details.This is quite divergent from the text for the 32-bit arm
KUSER_HELPERS.Could we please reuse that existing text as far as possible, with minimal changes?
e.g.
config
KUSER_HELPERS
bool "Enable kuser helpers for compat tasks" depends on COMPAT
default y
help
Warning: disabling this option may break user programs.Provide kuser helpers to compat tasks. The kernel provides helper code to userspace in read only form at a fixed location to allow userspace to be independent of the CPU type fitted to the system. This permits binaries to be run on ARMv4 through to ARMv8 without modification.
See
Documentation/arm/kernel_user_helpers.txtfor details.However, the fixed address nature of these helpers can be used by ROP (return orientated programming) authors when creating exploits.
If all of the binaries and libraries which run on your platform are built specifically for your platform, and make no use of these helpers, then you can turn this option off to hinder such exploits. However, in that case, if a binary or library relying on those helpers is run, it will not function correctly.
Say N here only if you are absolutely certain that you do not need these helpers; otherwise, the safe option is to say Y.
I believe that Will wanted to default the kuser helpers off, as applications using them need the instruction emulation which is also disabled by default.
I'm also not sure which binaries we expect to work on arm64, so I'm not sure if the ARMv4 reference is correct.
Thanks,
Mark.
Vincenzo Frascino
maintainerPATCH v5 9/23 arm64: compat: Add missing syscall numbers
In reply to Vincenzo FrascinovDSO requires gettimeofday and clock_gettime syscalls to implement the fallback mechanism.
Add the missing syscall numbers to unistd.h for arm64.
--- arch/arm64/include/asm/unistd.h | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/include/asm/unistd.h +3 -0
@@ -33,8 +33,11 @@#define __NR_compat_exit 1#define __NR_compat_read 3#define __NR_compat_write 4 +#define __NR_compat_gettimeofday 78#define __NR_compat_sigreturn 119#define __NR_compat_rt_sigreturn 173 +#define __NR_compat_clock_getres 247 +#define __NR_compat_clock_gettime 263/* * The following SVCs are ARM private.
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 10/23 arm64: compat: Expose signal related structures
In reply to Vincenzo FrascinoThe compat signal data structures are required as part of the compat vDSO implementation in order to provide the unwinding information for the sigreturn trampolines.
Expose the mentioned data structures as part of signal32.h.
--- arch/arm64/include/asm/signal32.h | 46 +++++++++++++++++++++++++++++++ arch/arm64/kernel/signal32.c | 46 ------------------------------- 2 files changed, 46 insertions(+), 46 deletions(-)
arch/arm64/include/asm/signal32.h +46 -0
@@ -20,6 +20,52 @@#ifdef CONFIG_COMPAT #include <linux/compat.h> +struct compat_sigcontext { +/* We always set these two fields to 0 */ +compat_ulong_t trap_no; +compat_ulong_t error_code; + +compat_ulong_t oldmask; +compat_ulong_t arm_r0; +compat_ulong_t arm_r1; +compat_ulong_t arm_r2; +compat_ulong_t arm_r3; +compat_ulong_t arm_r4; +compat_ulong_t arm_r5; +compat_ulong_t arm_r6; +compat_ulong_t arm_r7; +compat_ulong_t arm_r8; +compat_ulong_t arm_r9; +compat_ulong_t arm_r10; +compat_ulong_t arm_fp; +compat_ulong_t arm_ip; +compat_ulong_t arm_sp; +compat_ulong_t arm_lr; +compat_ulong_t arm_pc; +compat_ulong_t arm_cpsr; +compat_ulong_t fault_address; +}; + +struct compat_ucontext { +compat_ulong_t uc_flags; +compat_uptr_t uc_link; +compat_stack_t uc_stack; +struct compat_sigcontext uc_mcontext; +compat_sigset_t uc_sigmask; +int __unused[32 - (sizeof (compat_sigset_t) /sizeof (int ))]; +compat_ulong_t uc_regspace[128 ] __attribute__((__aligned__ (8 ))); +}; + +struct compat_sigframe { +struct compat_ucontext uc; +compat_ulong_t retcode[2 ]; +}; + +struct compat_rt_sigframe { +struct compat_siginfo info; +struct compat_sigframe sig; +}; +int compat_setup_frame (int usig,struct ksignal *ksig,sigset_t *set,struct pt_regs *regs); int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
arch/arm64/kernel/signal32.c +0 -46
58 lines not on this page · raw message
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 11/23 arm64: compat: Generate asm offsets for signals
In reply to Vincenzo FrascinoUpdate asm-offsets for arm64 to generate the correct offsets for compat signals.
They will be useful for the implementation of the compat sigreturn trampolines in vDSO context.
--- arch/arm64/include/asm/signal32.h | 14 +++++++++++++- arch/arm64/kernel/asm-offsets.c | 6 ++++++ 2 files changed, 19 insertions(+), 1 deletion(-)
arch/arm64/include/asm/signal32.h +13 -1
@@ -52,7 +52,7 @@ struct compat_ucontext {compat_stack_t uc_stack;struct compat_sigcontext uc_mcontext;compat_sigset_t uc_sigmask; -int __unused[32 - (sizeof (compat_sigset_t) /sizeof (int ))]; +int __unused[32 - (sizeof (compat_sigset_t) /sizeof (int ))]; compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8))); };
@@ -66,6 +66,18 @@ struct compat_rt_sigframe {struct compat_sigframe sig; }; +/* Macros for asm-offsets.c */ +#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \ + offsetof(struct compat_sigframe, uc) + \ + offsetof(struct compat_ucontext, uc_mcontext) + \ + offsetof(struct compat_sigcontext, arm_r0)) + +#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \ + offsetof(struct compat_rt_sigframe, sig) + \ + offsetof(struct compat_sigframe, uc) + \ + offsetof(struct compat_ucontext, uc_mcontext) + \ + offsetof(struct compat_sigcontext, arm_r0)) +int compat_setup_frame (int usig,struct ksignal *ksig,sigset_t *set,struct pt_regs *regs); int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
arch/arm64/kernel/asm-offsets.c +6 -0
@@ -30,6 +30,7 @@#include <asm/fixmap.h> #include <asm/thread_info.h> #include <asm/memory.h> +#include <asm/signal32.h> #include <asm/smp_plat.h> #include <asm/suspend.h> #include <linux/kbuild.h>
@@ -84,6 +85,11 @@ int main(void)DEFINE (S_STACKFRAME , offsetof(struct pt_regs , stackframe));DEFINE (S_FRAME_SIZE ,sizeof (struct pt_regs ));BLANK (); +#ifdef CONFIG_COMPAT +DEFINE (COMPAT_SIGFRAME_REGS_OFFSET ,OFFSET_OF_COMPAT_SIGFRAME_REGS ); +DEFINE (COMPAT_RT_SIGFRAME_REGS_OFFSET ,OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ); +BLANK (); +#endif DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));BLANK ();DEFINE (VMA_VM_MM , offsetof(struct vm_area_struct , vm_mm));
Signature · 1 line
2.20.1
Mark Rutland
reviewerRe: [PATCH v5 11/23] arm64: compat: Generate asm offsets for signals
arch/arm64/include/asm/signal32.h struct compat_ucontext { · line 55
Quoted 10 lines · Vincenzo Frascino
arch/arm64/include/asm/signal32.h +1
-1@@ -52,7 +52,7 @@ struct compat_ucontext {compat_stack_t uc_stack;struct compat_sigcontext uc_mcontext;compat_sigset_t uc_sigmask; -int __unused[32 - (sizeof (compat_sigset_t) /sizeof (int ))]; +int __unused[32 - (sizeof (compat_sigset_t) /sizeof (int ))];
Spurious whitespace change?
Quoted 28 lines
arch/arm64/include/asm/signal32.h +12
-0+/* Macros for asm-offsets.c */ +#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \ + offsetof(struct compat_sigframe, uc) + \ + offsetof(struct compat_ucontext, uc_mcontext) + \ + offsetof(struct compat_sigcontext, arm_r0)) + +#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \ + offsetof(struct compat_rt_sigframe, sig) + \ + offsetof(struct compat_sigframe, uc) + \ + offsetof(struct compat_ucontext, uc_mcontext) + \ + offsetof(struct compat_sigcontext, arm_r0)) +int compat_setup_frame (int usig,struct ksignal *ksig,sigset_t *set,struct pt_regs *regs); int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,arch/arm64/kernel/asm-offsets.c +1
-0@@ -30,6 +30,7 @@#include <asm/fixmap.h> #include <asm/thread_info.h> #include <asm/memory.h> +#include <asm/signal32.h> #include <asm/smp_plat.h> #include <asm/suspend.h> #include <linux/kbuild.h>
arch/arm64/kernel/asm-offsets.c int main(void) · line 90
Quoted 6 lines
arch/arm64/kernel/asm-offsets.c +3
-0@@ -84,6 +85,11 @@ int main(void)DEFINE (S_STACKFRAME , offsetof(struct pt_regs , stackframe));DEFINE (S_FRAME_SIZE ,sizeof (struct pt_regs ));BLANK (); +#ifdef CONFIG_COMPAT +DEFINE (COMPAT_SIGFRAME_REGS_OFFSET ,OFFSET_OF_COMPAT_SIGFRAME_REGS ); +DEFINE (COMPAT_RT_SIGFRAME_REGS_OFFSET ,OFFSET_OF_COMPAT_RT_SIGFRAME_REGS );
Can't we do:
DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_sigframe, uc.us_mcontext.arm_r0)
DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_rt_sigframe, sig.uc.us_mcontext.arm_r0)
... and avoid the OFFSET_OF_* macros entirely?
Thanks,
Mark.
Vincenzo Frascino
maintainerPATCH v5 12/23 lib: vdso: Add compat support
In reply to Vincenzo FrascinoSome 64 bit architectures have support for 32 bit applications that require a separate version of the vDSOs.
Add support to the generic code for compat fallback functions.
--- lib/vdso/gettimeofday.c | 4 ++++ 1 file changed, 4 insertions(+)
lib/vdso/gettimeofday.c +4 -0
@@ -20,7 +20,11 @@* - clock_gettime_fallback(): fallback for clock_gettime. * - clock_getres_fallback(): fallback for clock_getres. */ +#ifdef ENABLE_COMPAT_VDSO +#include <asm/vdso/compat_gettimeofday.h> +#else #include <asm/vdso/gettimeofday.h> +#endif /* ENABLE_COMPAT_VDSO */ static notrace int do_hres(const struct vdso_data *vd, clockid_t clk,
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 13/23 arm64: compat: Add vDSO
In reply to Vincenzo FrascinoProvide the arm64 compat (AArch32) vDSO in kernel/vdso32 in a similar way to what happens in kernel/vdso.
The compat vDSO leverages on an adaptation of the arm architecture code with few changes:
- Use of lib/vdso for gettimeofday
- Implementation of syscall based fallback
- Introduction of clock_getres for the compat library
- Implementation of trampolines
- Implementation of elf note
To build the compat vDSO a 32 bit compiler is required and needs to be specified via CONFIG_CROSS_COMPILE_COMPAT_VDSO.
The implementation of the configuration option will be contained in a future patch.
--- arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++++ .../include/asm/vdso/compat_gettimeofday.h | 119 +++++++++++ arch/arm64/kernel/vdso32/.gitignore | 2 + arch/arm64/kernel/vdso32/Makefile | 184 ++++++++++++++++++ arch/arm64/kernel/vdso32/note.c | 15 ++ arch/arm64/kernel/vdso32/sigreturn.S | 62 ++++++ arch/arm64/kernel/vdso32/vdso.S | 19 ++ arch/arm64/kernel/vdso32/vdso.lds.S | 81 ++++++++ arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++++ 9 files changed, 559 insertions(+) create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h create mode 100644 arch/arm64/kernel/vdso32/.gitignore create mode 100644 arch/arm64/kernel/vdso32/Makefile create mode 100644 arch/arm64/kernel/vdso32/note.c create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S create mode 100644 arch/arm64/kernel/vdso32/vdso.S create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
arch/arm64/include/asm/vdso/compat_barrier.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (C) 2018 ARM Limited +*/ +#ifndef __COMPAT_BARRIER_H +#define __COMPAT_BARRIER_H + +#include <asm/barrier.h> + +#if __LINUX_ARM_ARCH__ >=8 +#define aarch32_smp_mb () dmb(ish) +#define aarch32_smp_rmb () dmb(ishld) +#define aarch32_smp_wmb () dmb(ishst) +#else +#define aarch32_smp_mb () dmb(ish) +#define aarch32_smp_rmb () aarch32_smp_mb() +#define aarch32_smp_wmb () dmb(ishst) +#endif + +/* +* Warning: This code is meant to be used with +* ENABLE_COMPAT_VDSO only. +*/ +#ifndef ENABLE_COMPAT_VDSO +#error This header is meant to be used with ENABLE_COMPAT_VDSO only +#endif + +#undef smp_mb +#undef smp_rmb +#undef smp_wmb + +#define smp_mb () aarch32_smp_mb() +#define smp_rmb () aarch32_smp_rmb() +#define smp_wmb () aarch32_smp_wmb() + +#endif /* __COMPAT_BARRIER_H */
arch/arm64/include/asm/vdso/compat_gettimeofday.h new file +119 -0
119 lines not on this page · raw message
arch/arm64/kernel/vdso32/.gitignore new file +2 -0
@@ -0,0 +1,2 @@ +vdso.lds +vdso.so.raw
arch/arm64/kernel/vdso32/Makefile new file +184 -0
184 lines not on this page · raw message
arch/arm64/kernel/vdso32/note.c new file +15 -0
@@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (C) 2012-2018 ARM Limited +* +* This supplies .note.* sections to go into the PT_NOTE inside the vDSO text. +* Here we can supply some information useful to userland. +*/ + +#include <linux/uts.h> +#include <linux/version.h> +#include <linux/elfnote.h> +#include <linux/build-salt.h> + +ELFNOTE32 ("Linux" ,0 ,LINUX_VERSION_CODE ); +BUILD_SALT ;
arch/arm64/kernel/vdso32/sigreturn.S new file +62 -0
62 lines not on this page · raw message
arch/arm64/kernel/vdso32/vdso.S new file +19 -0
@@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2012 ARM Limited + */ + +#include <linux/init.h> +#include <linux/linkage.h> +#include <linux/const.h> +#include <asm/page.h> + + .globl vdso32_start, vdso32_end + .section .rodata + .balign PAGE_SIZE +vdso32_start: + .incbin "arch/arm64/kernel/vdso32/vdso.so" + .balign PAGE_SIZE +vdso32_end: + + .previous
arch/arm64/kernel/vdso32/vdso.lds.S new file +81 -0
81 lines not on this page · raw message
arch/arm64/kernel/vdso32/vgettimeofday.c new file +41 -0
41 lines not on this page · raw message
Signature · 1 line
2.20.1
Mark Rutland
reviewerRe: [PATCH v5 13/23] arm64: compat: Add vDSO
Quoted 6 lines · Vincenzo Frascino
arch/arm64/include/asm/vdso/compat_barrier.h new file +0
-0
arch/arm64/include/asm/vdso/compat_barrier.h line 8
Quoted 8 lines
arch/arm64/include/asm/vdso/compat_barrier.h new file +8
-0@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (C) 2018 ARM Limited +*/ +#ifndef __COMPAT_BARRIER_H +#define __COMPAT_BARRIER_H + +#include <asm/barrier.h>
I think it would be safer to not include this header, and explicitly define the AArch32 barriers here in isolation.
arch/arm64/include/asm/vdso/compat_barrier.h line 18
Quoted 10 lines
arch/arm64/include/asm/vdso/compat_barrier.h new file +10
-0+ +#if __LINUX_ARM_ARCH__ >=8 +#define aarch32_smp_mb () dmb(ish) +#define aarch32_smp_rmb () dmb(ishld) +#define aarch32_smp_wmb () dmb(ishst) +#else +#define aarch32_smp_mb () dmb(ish) +#define aarch32_smp_rmb () aarch32_smp_mb() +#define aarch32_smp_wmb () dmb(ishst) +#endif
Can't we just code an AArch32-specific version of dmb() here? AFAICT that's the only reason we include the native barrier.h.
Quoted 5 lines
arch/arm64/include/asm/vdso/compat_barrier.h new file +5
-0+/* +* Warning: This code is meant to be used with +* ENABLE_COMPAT_VDSO only. +*/ +#ifndef ENABLE_COMPAT_VDSO
arch/arm64/include/asm/vdso/compat_barrier.h line 34
Quoted 10 lines
arch/arm64/include/asm/vdso/compat_barrier.h new file +10
-0+#error This header is meant to be used with ENABLE_COMPAT_VDSO only +#endif + +#undef smp_mb +#undef smp_rmb +#undef smp_wmb + +#define smp_mb () aarch32_smp_mb() +#define smp_rmb () aarch32_smp_rmb() +#define smp_wmb () aarch32_smp_wmb()
This undeffing looks very fragile to me.
[...]
Quoted 15 lines
arch/arm64/kernel/vdso32/.gitignore new file +2
-0@@ -0,0 +1,2 @@ +vdso.lds +vdso.so.rawarch/arm64/kernel/vdso32/Makefile new file +1
-0@@ -0,0 +1,184 @@ +# SPDX-License-Identifier: GPL-2.0
arch/arm64/kernel/vdso32/Makefile line 11
Quoted 10 lines
arch/arm64/kernel/vdso32/Makefile new file +10
-0+# +# Makefile for vdso32 +# + +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before +# the inclusion of generic Makefile. +ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32 +include $(srctree)/lib/vdso/Makefile + +CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
To match HOSTCC, it might make sense to call this COMPATCC
... and likewise, CROSS_COMPILE_COMPAT
Thanks,
Mark.
Vincenzo Frascino
maintainerPATCH v5 14/23 arm64: Refactor vDSO code
In reply to Vincenzo FrascinoMost of the code for initializing the vDSOs in arm64 and compat will be in common, hence a refactor of the current code is required to avoid duplication and simplify maintainability.
Refactor vdso.c to simplify the implementation of arm64 vDSO compat (which will be pushed with a future patch).
--- arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++-------------- 1 file changed, 135 insertions(+), 73 deletions(-)
arch/arm64/kernel/vdso.c +135 -73
@@ -41,7 +41,30 @@#include <asm/vdso.h> extern char vdso_start[], vdso_end[]; -static unsigned long vdso_pages__ro_after_init ; + +/* vdso_lookup arch_index */ +enum arch_vdso_type { +ARM64_VDSO =0 , +}; + +struct __vdso_lookup_t { +const char *name; +const char *vdso_code_start; +const char *vdso_code_end; +unsigned long vdso_pages; +/* Data Mapping */ +struct vm_special_mapping *dm; +/* Code Mapping */ +struct vm_special_mapping *cm; +}; + +static struct __vdso_lookup_t vdso_lookup[2 ]__ro_after_init = { + { + .name ="vdso" , + .vdso_code_start = vdso_start, + .vdso_code_end = vdso_end, + }, +};/* * The vDSO data page.
@@ -52,6 +75,106 @@ static union { }vdso_data_store __page_aligned_data;struct vdso_data *vdso_data = &vdso_data_store.data; +static int __vdso_remap (enum arch_vdso_type arch_index, +const struct vm_special_mapping *sm, +struct vm_area_struct *new_vma) +{ +unsigned long new_size = new_vma->vm_end - new_vma->vm_start; +unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end - + vdso_lookup[arch_index].vdso_code_start; + +if (vdso_size != new_size) +return -EINVAL ; + + current->mm->context.vdso = (void *)new_vma->vm_start; + +return 0 ; +} + +static int __vdso_init (enum arch_vdso_type arch_index) +{ +int i; +struct page **vdso_pagelist; +unsigned long pfn; + +if (memcmp (vdso_lookup[arch_index].vdso_code_start,"\177ELF" ,4 )) { +pr_err ("vDSO is not a valid ELF object!\n" ); +return -EINVAL ; + } + + vdso_lookup[arch_index].vdso_pages = ( + vdso_lookup[arch_index].vdso_code_end - + vdso_lookup[arch_index].vdso_code_start) >> +PAGE_SHIFT ; +pr_info ("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n" , + vdso_lookup[arch_index].name, + vdso_lookup[arch_index].vdso_pages +1 , + vdso_lookup[arch_index].vdso_pages, + vdso_lookup[arch_index].vdso_code_start,1L , vdso_data); + +/* Allocate the vDSO pagelist, plus a page for the data. */ + vdso_pagelist =kcalloc (vdso_lookup[arch_index].vdso_pages +1 , +sizeof (struct page *), +GFP_KERNEL ); +if (vdso_pagelist ==NULL ) +return -ENOMEM ; + +/* Grab the vDSO data page. */ + vdso_pagelist[0 ] =phys_to_page (__pa_symbol (vdso_data)); + + +/* Grab the vDSO code pages. */ + pfn =sym_to_pfn (vdso_lookup[arch_index].vdso_code_start); + +for (i =0 ; i < vdso_lookup[arch_index].vdso_pages; i++) + vdso_pagelist[i +1 ] =pfn_to_page (pfn + i); + + vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0 ]; + vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1 ]; + +return 0 ; +} + +static int __setup_additional_pages (enum arch_vdso_type arch_index, +struct mm_struct *mm, +struct linux_binprm *bprm, +int uses_interp) +{ +unsigned long vdso_base, vdso_text_len, vdso_mapping_len; +void *ret; + + vdso_text_len = vdso_lookup[arch_index].vdso_pages <<PAGE_SHIFT ; +/* Be sure to map the data page */ + vdso_mapping_len = vdso_text_len +PAGE_SIZE ; + + vdso_base =get_unmapped_area (NULL ,0 , vdso_mapping_len,0 ,0 ); +if (IS_ERR_VALUE (vdso_base)) { + ret =ERR_PTR (vdso_base); + goto up_fail; + } + + ret =_install_special_mapping (mm, vdso_base,PAGE_SIZE , +VM_READ |VM_MAYREAD , + vdso_lookup[arch_index].dm); +if (IS_ERR (ret)) + goto up_fail; + + vdso_base +=PAGE_SIZE ; + mm->context.vdso = (void *)vdso_base; + ret =_install_special_mapping (mm, vdso_base, vdso_text_len, +VM_READ |VM_EXEC | +VM_MAYREAD |VM_MAYWRITE |VM_MAYEXEC , + vdso_lookup[arch_index].cm); +if (IS_ERR (ret)) + goto up_fail; + +return 0 ; + +up_fail: + mm->context.vdso =NULL ; +return PTR_ERR (ret); +} +#ifdef CONFIG_COMPAT /* * Create and map the vectors page for AArch32 tasks.
@@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;* 1 - sigreturn code */ static struct page *aarch32_vdso_pages[2 ]__ro_after_init ; -static const struct vm_special_mapping aarch32_vdso_spec[2 ] = { +static struct vm_special_mapping aarch32_vdso_spec[2 ]__ro_after_init = { {/* Must be named [vectors] for compatibility with arm. */ .name ="[vectors]" ,
@@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)static int vdso_mremap (const struct vm_special_mapping *sm,struct vm_area_struct *new_vma) { -unsigned long new_size = new_vma->vm_end - new_vma->vm_start; -unsigned long vdso_size = vdso_end - vdso_start; - -if (vdso_size != new_size) -return -EINVAL ; - - current->mm->context.vdso = (void *)new_vma->vm_start; - -return 0 ; +return __vdso_remap (ARM64_VDSO , sm, new_vma); }static struct vm_special_mapping vdso_spec[2 ]__ro_after_init = {
@@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {static int __init vdso_init (void ) { -int i; -struct page **vdso_pagelist; -unsigned long pfn; + vdso_lookup[ARM64_VDSO ].dm = &vdso_spec[0 ]; + vdso_lookup[ARM64_VDSO ].cm = &vdso_spec[1 ]; -if (memcmp (vdso_start,"\177ELF" ,4 )) { -pr_err ("vDSO is not a valid ELF object!\n" ); -return -EINVAL ; - } - - vdso_pages = (vdso_end - vdso_start) >>PAGE_SHIFT ; -pr_info ("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n" , - vdso_pages +1 , vdso_pages, vdso_start,1L , vdso_data); - -/* Allocate the vDSO pagelist, plus a page for the data. */ - vdso_pagelist =kcalloc (vdso_pages +1 ,sizeof (struct page *), -GFP_KERNEL ); -if (vdso_pagelist ==NULL ) -return -ENOMEM ; - -/* Grab the vDSO data page. */ - vdso_pagelist[0 ] =phys_to_page (__pa_symbol (vdso_data)); - - -/* Grab the vDSO code pages. */ - pfn =sym_to_pfn (vdso_start); - -for (i =0 ; i < vdso_pages; i++) - vdso_pagelist[i +1 ] =pfn_to_page (pfn + i); - - vdso_spec[0 ].pages = &vdso_pagelist[0 ]; - vdso_spec[1 ].pages = &vdso_pagelist[1 ]; - -return 0 ; +return __vdso_init (ARM64_VDSO ); }arch_initcall (vdso_init);
@@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) {struct mm_struct *mm = current->mm; -unsigned long vdso_base, vdso_text_len, vdso_mapping_len; -void *ret; - - vdso_text_len = vdso_pages <<PAGE_SHIFT ; -/* Be sure to map the data page */ - vdso_mapping_len = vdso_text_len +PAGE_SIZE ; +int ret;if (down_write_killable (&mm->mmap_sem))return -EINTR ; - vdso_base =get_unmapped_area (NULL ,0 , vdso_mapping_len,0 ,0 ); -if (IS_ERR_VALUE (vdso_base)) { - ret =ERR_PTR (vdso_base); - goto up_fail; - } - ret =_install_special_mapping (mm, vdso_base,PAGE_SIZE , -VM_READ |VM_MAYREAD , - &vdso_spec[0 ]); -if (IS_ERR (ret)) - goto up_fail; - - vdso_base +=PAGE_SIZE ; - mm->context.vdso = (void *)vdso_base; - ret =_install_special_mapping (mm, vdso_base, vdso_text_len, -VM_READ |VM_EXEC | -VM_MAYREAD |VM_MAYWRITE |VM_MAYEXEC , - &vdso_spec[1 ]); -if (IS_ERR (ret)) - goto up_fail; + ret =__setup_additional_pages (ARM64_VDSO , + mm, + bprm, + uses_interp);up_write (&mm->mmap_sem); -return 0 ; -up_fail: - mm->context.vdso =NULL ; -up_write (&mm->mmap_sem); -return PTR_ERR (ret); +return ret; }#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
Signature · 1 line
2.20.1
Mark Rutland
reviewerRe: [PATCH v5 14/23] arm64: Refactor vDSO code
Quoted 20 lines · Vincenzo Frascino
Most of the code for initializing the vDSOs in arm64 and compat will be in common, hence a refactor of the current code is required to avoid duplication and simplify maintainability.
Refactor
vdso.cto simplify the implementation of arm64 vDSO compat (which will be pushed with a future patch).Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Signed-off-by Vincenzo Frascino <[email protected]>--- arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++-------------- 1 file changed, 135 insertions(+), 73 deletions(-)arch/arm64/kernel/vdso.c +0
-0@@ -41,7 +41,30 @@#include <asm/vdso.h>
arch/arm64/kernel/vdso.c line 50
Quoted 10 lines
arch/arm64/kernel/vdso.c +7
-1extern char vdso_start[], vdso_end[]; -static unsigned long vdso_pages__ro_after_init ; + +/* vdso_lookup arch_index */ +enum arch_vdso_type { +ARM64_VDSO =0 , +}; + +struct __vdso_lookup_t {
If you want to give this a _t suffix, please use a typedef so that you don't need to also say 'struct' to use it.
I think this would be better named as struct vdso_instance, or struct vdso_abi.
Thanks,
Mark.
Quoted 249 lines
arch/arm64/kernel/vdso.c +128
-72+const char *name; +const char *vdso_code_start; +const char *vdso_code_end; +unsigned long vdso_pages; +/* Data Mapping */ +struct vm_special_mapping *dm; +/* Code Mapping */ +struct vm_special_mapping *cm; +}; + +static struct __vdso_lookup_t vdso_lookup[2 ]__ro_after_init = { + { + .name ="vdso" , + .vdso_code_start = vdso_start, + .vdso_code_end = vdso_end, + }, +};/* * The vDSO data page. @@ -52,6 +75,106 @@ static union { }vdso_data_store __page_aligned_data;struct vdso_data *vdso_data = &vdso_data_store.data; +static int __vdso_remap (enum arch_vdso_type arch_index, +const struct vm_special_mapping *sm, +struct vm_area_struct *new_vma) +{ +unsigned long new_size = new_vma->vm_end - new_vma->vm_start; +unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end - + vdso_lookup[arch_index].vdso_code_start; + +if (vdso_size != new_size) +return -EINVAL ; + + current->mm->context.vdso = (void *)new_vma->vm_start; + +return 0 ; +} + +static int __vdso_init (enum arch_vdso_type arch_index) +{ +int i; +struct page **vdso_pagelist; +unsigned long pfn; + +if (memcmp (vdso_lookup[arch_index].vdso_code_start,"\177ELF" ,4 )) { +pr_err ("vDSO is not a valid ELF object!\n" ); +return -EINVAL ; + } + + vdso_lookup[arch_index].vdso_pages = ( + vdso_lookup[arch_index].vdso_code_end - + vdso_lookup[arch_index].vdso_code_start) >> +PAGE_SHIFT ; +pr_info ("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n" , + vdso_lookup[arch_index].name, + vdso_lookup[arch_index].vdso_pages +1 , + vdso_lookup[arch_index].vdso_pages, + vdso_lookup[arch_index].vdso_code_start,1L , vdso_data); + +/* Allocate the vDSO pagelist, plus a page for the data. */ + vdso_pagelist =kcalloc (vdso_lookup[arch_index].vdso_pages +1 , +sizeof (struct page *), +GFP_KERNEL ); +if (vdso_pagelist ==NULL ) +return -ENOMEM ; + +/* Grab the vDSO data page. */ + vdso_pagelist[0 ] =phys_to_page (__pa_symbol (vdso_data)); + + +/* Grab the vDSO code pages. */ + pfn =sym_to_pfn (vdso_lookup[arch_index].vdso_code_start); + +for (i =0 ; i < vdso_lookup[arch_index].vdso_pages; i++) + vdso_pagelist[i +1 ] =pfn_to_page (pfn + i); + + vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0 ]; + vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1 ]; + +return 0 ; +} + +static int __setup_additional_pages (enum arch_vdso_type arch_index, +struct mm_struct *mm, +struct linux_binprm *bprm, +int uses_interp) +{ +unsigned long vdso_base, vdso_text_len, vdso_mapping_len; +void *ret; + + vdso_text_len = vdso_lookup[arch_index].vdso_pages <<PAGE_SHIFT ; +/* Be sure to map the data page */ + vdso_mapping_len = vdso_text_len +PAGE_SIZE ; + + vdso_base =get_unmapped_area (NULL ,0 , vdso_mapping_len,0 ,0 ); +if (IS_ERR_VALUE (vdso_base)) { + ret =ERR_PTR (vdso_base); + goto up_fail; + } + + ret =_install_special_mapping (mm, vdso_base,PAGE_SIZE , +VM_READ |VM_MAYREAD , + vdso_lookup[arch_index].dm); +if (IS_ERR (ret)) + goto up_fail; + + vdso_base +=PAGE_SIZE ; + mm->context.vdso = (void *)vdso_base; + ret =_install_special_mapping (mm, vdso_base, vdso_text_len, +VM_READ |VM_EXEC | +VM_MAYREAD |VM_MAYWRITE |VM_MAYEXEC , + vdso_lookup[arch_index].cm); +if (IS_ERR (ret)) + goto up_fail; + +return 0 ; + +up_fail: + mm->context.vdso =NULL ; +return PTR_ERR (ret); +} +#ifdef CONFIG_COMPAT /* * Create and map the vectors page for AArch32 tasks. @@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;* 1 - sigreturn code */ static struct page *aarch32_vdso_pages[2 ]__ro_after_init ; -static const struct vm_special_mapping aarch32_vdso_spec[2 ] = { +static struct vm_special_mapping aarch32_vdso_spec[2 ]__ro_after_init = { {/* Must be named [vectors] for compatibility with arm. */ .name ="[vectors]" ,@@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)static int vdso_mremap (const struct vm_special_mapping *sm,struct vm_area_struct *new_vma) { -unsigned long new_size = new_vma->vm_end - new_vma->vm_start; -unsigned long vdso_size = vdso_end - vdso_start; - -if (vdso_size != new_size) -return -EINVAL ; - - current->mm->context.vdso = (void *)new_vma->vm_start; - -return 0 ; +return __vdso_remap (ARM64_VDSO , sm, new_vma); }static struct vm_special_mapping vdso_spec[2 ]__ro_after_init = {@@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {static int __init vdso_init (void ) { -int i; -struct page **vdso_pagelist; -unsigned long pfn; + vdso_lookup[ARM64_VDSO ].dm = &vdso_spec[0 ]; + vdso_lookup[ARM64_VDSO ].cm = &vdso_spec[1 ]; -if (memcmp (vdso_start,"\177ELF" ,4 )) { -pr_err ("vDSO is not a valid ELF object!\n" ); -return -EINVAL ; - } - - vdso_pages = (vdso_end - vdso_start) >>PAGE_SHIFT ; -pr_info ("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n" , - vdso_pages +1 , vdso_pages, vdso_start,1L , vdso_data); - -/* Allocate the vDSO pagelist, plus a page for the data. */ - vdso_pagelist =kcalloc (vdso_pages +1 ,sizeof (struct page *), -GFP_KERNEL ); -if (vdso_pagelist ==NULL ) -return -ENOMEM ; - -/* Grab the vDSO data page. */ - vdso_pagelist[0 ] =phys_to_page (__pa_symbol (vdso_data)); - - -/* Grab the vDSO code pages. */ - pfn =sym_to_pfn (vdso_start); - -for (i =0 ; i < vdso_pages; i++) - vdso_pagelist[i +1 ] =pfn_to_page (pfn + i); - - vdso_spec[0 ].pages = &vdso_pagelist[0 ]; - vdso_spec[1 ].pages = &vdso_pagelist[1 ]; - -return 0 ; +return __vdso_init (ARM64_VDSO ); }arch_initcall (vdso_init);@@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) {struct mm_struct *mm = current->mm; -unsigned long vdso_base, vdso_text_len, vdso_mapping_len; -void *ret; - - vdso_text_len = vdso_pages <<PAGE_SHIFT ; -/* Be sure to map the data page */ - vdso_mapping_len = vdso_text_len +PAGE_SIZE ; +int ret;if (down_write_killable (&mm->mmap_sem))return -EINTR ; - vdso_base =get_unmapped_area (NULL ,0 , vdso_mapping_len,0 ,0 ); -if (IS_ERR_VALUE (vdso_base)) { - ret =ERR_PTR (vdso_base); - goto up_fail; - } - ret =_install_special_mapping (mm, vdso_base,PAGE_SIZE , -VM_READ |VM_MAYREAD , - &vdso_spec[0 ]); -if (IS_ERR (ret)) - goto up_fail; - - vdso_base +=PAGE_SIZE ; - mm->context.vdso = (void *)vdso_base; - ret =_install_special_mapping (mm, vdso_base, vdso_text_len, -VM_READ |VM_EXEC | -VM_MAYREAD |VM_MAYWRITE |VM_MAYEXEC , - &vdso_spec[1 ]); -if (IS_ERR (ret)) - goto up_fail; + ret =__setup_additional_pages (ARM64_VDSO , + mm, + bprm, + uses_interp);up_write (&mm->mmap_sem); -return 0 ; -up_fail: - mm->context.vdso =NULL ; -up_write (&mm->mmap_sem); -return PTR_ERR (ret); +return ret; }#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 15/23 arm64: compat: vDSO setup for compat layer
In reply to Vincenzo FrascinoIf CONFIG_GENERIC_COMPAT_VDSO is enabled, compat vDSO are installed in a compat (32 bit) process instead of sigpage.
Add the necessary code to setup the vDSO required pages.
--- arch/arm64/kernel/vdso.c | 60 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-)
arch/arm64/kernel/vdso.c +59 -1
@@ -41,10 +41,16 @@#include <asm/vdso.h> extern char vdso_start[], vdso_end[]; +#ifdef CONFIG_COMPAT_VDSO +extern char vdso32_start[], vdso32_end[]; +#endif /* CONFIG_COMPAT_VDSO */ /* vdso_lookup arch_index */ enum arch_vdso_type {ARM64_VDSO =0 , +#ifdef CONFIG_COMPAT_VDSO +ARM64_VDSO32 =1 , +#endif /* CONFIG_COMPAT_VDSO */ };struct __vdso_lookup_t {
@@ -64,6 +70,13 @@ static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = { .vdso_code_start = vdso_start, .vdso_code_end = vdso_end, }, +#ifdef CONFIG_COMPAT_VDSO + { + .name = "vdso32", + .vdso_code_start = vdso32_start, + .vdso_code_end = vdso32_end, + }, +#endif /* CONFIG_COMPAT_VDSO */ };/*
@@ -179,23 +192,45 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,/* * Create and map the vectors page for AArch32 tasks. */ +#ifdef CONFIG_COMPAT_VDSO +static int aarch32_vdso_mremap (const struct vm_special_mapping *sm, +struct vm_area_struct *new_vma) +{ +return __vdso_remap (ARM64_VDSO32 , sm, new_vma); +} +#endif /* CONFIG_COMPAT_VDSO */ +/* * aarch32_vdso_pages: * 0 - kuser helpers * 1 - sigreturn code +* or (CONFIG_COMPAT_VDSO): +* 0 - kuser helpers +* 1 - vdso data +* 2 - vdso code */ static struct page *aarch32_vdso_pages[2 ]__ro_after_init ; -static struct vm_special_mapping aarch32_vdso_spec[2 ]__ro_after_init = { +static struct vm_special_mapping aarch32_vdso_spec[3 ]__ro_after_init = { {/* Must be named [vectors] for compatibility with arm. */ .name ="[vectors]" , .pages = &aarch32_vdso_pages[0 ], }, +#ifdef CONFIG_COMPAT_VDSO + { + .name ="[vvar]" , + }, + { + .name ="[vdso]" , + .mremap = aarch32_vdso_mremap, + }, +#else {/* Must be named [sigpage] for compatibility with arm. */ .name ="[sigpage]" , .pages = &aarch32_vdso_pages[1 ], }, +#endif /* CONFIG_COMPAT_VDSO */ };#ifdef CONFIG_KUSER_HELPERS
@@ -227,6 +262,15 @@ static int aarch32_alloc_kuser_vdso_page(void) }#endif /* CONFIG_KUSER_HELPER */ +#ifdef CONFIG_COMPAT_VDSO +static int aarch32_vdso_init (void ) +{ + vdso_lookup[ARM64_VDSO32 ].dm = &aarch32_vdso_spec[1 ]; + vdso_lookup[ARM64_VDSO32 ].cm = &aarch32_vdso_spec[2 ]; + +return __vdso_init (ARM64_VDSO32 ); +} +#else static int aarch32_alloc_sigreturn_vdso_page (void ) {extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@@ -249,11 +293,16 @@ static int aarch32_alloc_sigreturn_vdso_page(void)return 0 ; } +#endif /* CONFIG_COMPAT_VDSO */ static int __init aarch32_alloc_vdso_pages (void ) { return aarch32_alloc_kuser_vdso_page() & +#ifdef CONFIG_COMPAT_VDSO +aarch32_vdso_init (); +#else aarch32_alloc_sigreturn_vdso_page (); +#endif /* CONFIG_COMPAT_VDSO */ }arch_initcall (aarch32_alloc_vdso_pages);
@@ -278,6 +327,7 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm) }#endif /* CONFIG_KUSER_HELPERS */ +#ifndef CONFIG_COMPAT_VDSO static int aarch32_sigreturn_setup (struct mm_struct *mm) {unsigned long addr;
@@ -301,6 +351,7 @@ static int aarch32_sigreturn_setup(struct mm_struct *mm) out: return PTR_ERR_OR_ZERO(ret); } +#endif /* !CONFIG_COMPAT_VDSO */ int aarch32_setup_additional_pages (struct linux_binprm *bprm,int uses_interp) {
@@ -314,7 +365,14 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)if (ret) goto out; +#ifdef CONFIG_COMPAT_VDSO + ret =__setup_additional_pages (ARM64_VDSO32 , + mm, + bprm, + uses_interp); +#else ret =aarch32_sigreturn_setup (mm); +#endif /* CONFIG_COMPAT_VDSO */ out:up_write (&mm->mmap_sem);
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 16/23 arm64: elf: vDSO code page discovery
In reply to Vincenzo FrascinoLike in normal vDSOs, when compat vDSOs are enabled the auxiliary vector symbol AT_SYSINFO_EHDR needs to point at the address of the vDSO code, to allow the dynamic linker to find it.
Add the necessary code to the elf arm64 module to make this possible.
--- arch/arm64/include/asm/elf.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
arch/arm64/include/asm/elf.h +14 -0
@@ -213,7 +213,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG]; ({ \set_thread_flag (TIF_32BIT ); \ }) +#ifdef CONFIG_GENERIC_COMPAT_VDSO +#define COMPAT_ARCH_DLINFO \ +do { \ +/* \ +* Note that we use Elf64_Off instead of elf_addr_t because \ +* elf_addr_t in compat is defined as Elf32_Addr and casting \ +* current->mm->context.vdso to it triggers a cast warning of \ +* cast from pointer to integer of different size. \ +*/ \ +NEW_AUX_ENT (AT_SYSINFO_EHDR , \ + (Elf64_Off )current->mm->context.vdso); \ +} while (0) +#else#define COMPAT_ARCH_DLINFO +#endif extern int aarch32_setup_additional_pages (struct linux_binprm *bprm,int uses_interp);#define compat_arch_setup_additional_pages \
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 17/23 arm64: compat: Get sigreturn trampolines from vDSO
In reply to Vincenzo FrascinoWhen the compat vDSO is enabled, the sigreturn trampolines are not anymore available through [sigpage] but through [vdso].
Add the relevant code the enable the feature.
--- arch/arm64/include/asm/vdso.h | 3 +++ arch/arm64/kernel/signal32.c | 29 +++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 2 deletions(-)
arch/arm64/include/asm/vdso.h +3 -0
@@ -28,6 +28,9 @@#ifndef __ASSEMBLY__#include <generated/vdso-offsets.h> +#ifdef CONFIG_COMPAT_VDSO +#include <generated/vdso32-offsets.h> +#endif #define VDSO_SYMBOL (base, name) \ ({ \
arch/arm64/kernel/signal32.c +27 -2
@@ -29,6 +29,7 @@#include <asm/traps.h> #include <linux/uaccess.h> #include <asm/unistd.h> +#include <asm/vdso.h> struct compat_vfp_sigframe {compat_ulong_t magic;
@@ -333,7 +334,6 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,compat_ulong_t retcode;compat_ulong_t spsr = regs->pstate & ~(PSR_f |PSR_AA32_E_BIT );int thumb; -void *sigreturn_base;/* Check if the handler is written for ARM or Thumb */ thumb = handler &1 ;
@@ -353,14 +353,39 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka, retcode =ptr_to_compat (ka->sa.sa_restorer); }else {/* Set up sigreturn pointer */ +#ifdef CONFIG_COMPAT_VDSO +void *vdso_base = current->mm->context.vdso; +void *vdso_trampoline; + +if (ka->sa.sa_flags &SA_SIGINFO ) { +if (thumb) { + vdso_trampoline =VDSO_SYMBOL (vdso_base, + compat_rt_sigreturn_thumb); + }else { + vdso_trampoline =VDSO_SYMBOL (vdso_base, + compat_rt_sigreturn_arm); + } + }else { +if (thumb) { + vdso_trampoline =VDSO_SYMBOL (vdso_base, + compat_sigreturn_thumb); + }else { + vdso_trampoline =VDSO_SYMBOL (vdso_base, + compat_sigreturn_arm); + } + } + + retcode =ptr_to_compat (vdso_trampoline) + thumb; +#else unsigned int idx = thumb <<1 ; - sigreturn_base = current->mm->context.vdso; +void *sigreturn_base = current->mm->context.vdso;if (ka->sa.sa_flags &SA_SIGINFO ) idx +=3 ; retcode =ptr_to_compat (sigreturn_base) + (idx <<2 ) + thumb; +#endif } regs->regs[0 ] = usig;
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 18/23 arm64: Add vDSO compat support
In reply to Vincenzo FrascinoAdd vDSO compat support to the arm64 building system.
--- arch/arm64/Kconfig | 1 + arch/arm64/Makefile | 23 +++++++++++++++++++++-- arch/arm64/kernel/Makefile | 6 +++++- 3 files changed, 27 insertions(+), 3 deletions(-)
arch/arm64/Kconfig +1 -0
@@ -101,6 +101,7 @@ config ARM64 select GENERIC_STRNLEN_USER select GENERIC_TIME_VSYSCALL select GENERIC_GETTIMEOFDAY + select GENERIC_COMPAT_VDSO select HANDLE_DOMAIN_IRQ select HARDIRQS_SW_RESEND select HAVE_PCI
arch/arm64/Makefile +21 -2
@@ -49,9 +49,25 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable) endif endif -KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) +ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y) + CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%) + + ifeq ($(CONFIG_CC_IS_CLANG), y) + $(warning CROSS_COMPILE_ARM32 is clang, the compat vDSO will not be built) + else ifeq ($(CROSS_COMPILE_ARM32),) + $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built) + else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),) + $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32) + else + export CROSS_COMPILE_ARM32 + export CONFIG_COMPAT_VDSO := y + compat_vdso := -DCONFIG_COMPAT_VDSO=1 + endif +endif + +KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso) KBUILD_CFLAGS += -fno-asynchronous-unwind-tables -KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) +KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso) KBUILD_CFLAGS += $(call cc-option,-mabi=lp64) KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@@ -163,6 +179,9 @@ ifeq ($(KBUILD_EXTMOD),) prepare: vdso_prepare vdso_prepare: prepare0 $(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h + $(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \ + $(build)=arch/arm64/kernel/vdso32 \ + include/generated/vdso32-offsets.h) endif define archhelp
arch/arm64/kernel/Makefile +5 -1
@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE $(call if_changed,objcopy) obj-$(CONFIG_COMPAT) += sys32.o signal32.o \ - sigreturn32.o sys_compat.o + sys_compat.o +ifneq ($(CONFIG_COMPAT_VDSO), y) +obj-$(CONFIG_COMPAT) += sigreturn32.o +endif obj-$(CONFIG_KUSER_HELPERS) += kuser32.o obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o obj-$(CONFIG_MODULES) += module.o
@@ -62,6 +65,7 @@ obj-$(CONFIG_ARM64_SSBD) += ssbd.o obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o obj-y += vdso/ probes/ +obj-$(CONFIG_COMPAT_VDSO) += vdso32/ head-y := head.o extra-y += $(head-y) vmlinux.lds
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 19/23 arm64: Enable compat vDSO support
In reply to Vincenzo FrascinoEnable compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-. For this to take effect and for the compat vDSO to compile correctly, it is required that this compiler is in the PATH.
--- arch/arm64/configs/defconfig | 1 + 1 file changed, 1 insertion(+)
arch/arm64/configs/defconfig +1 -0
@@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y CONFIG_ACPI_APEI_EINJ=y CONFIG_VIRTUALIZATION=y CONFIG_KVM=y +CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-" CONFIG_ARM64_CRYPTO=y CONFIG_CRYPTO_SHA1_ARM64_CE=y CONFIG_CRYPTO_SHA2_ARM64_CE=y
Signature · 1 line
2.20.1
Mark Rutland
reviewerRe: [PATCH v5 19/23] arm64: Enable compat vDSO support
Quoted 5 lines · Vincenzo Frascino
Enable compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-. For this to take effect and for the compat vDSO to compile correctly, it is required that this compiler is in the PATH.
I do not have such a compiler in my path by default.
I hope that we handle that gracefully, rather than causing a build failure.
Can't we detect the presence of the compat compiler at config time, as happens for the native compiler?
Thanks,
Mark.
Quoted 23 lines
Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Signed-off-by Vincenzo Frascino <[email protected]>--- arch/arm64/configs/defconfig | 1 + 1 file changed, 1 insertion(+)arch/arm64/configs/defconfig +1
-0@@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y CONFIG_ACPI_APEI_EINJ=y CONFIG_VIRTUALIZATION=y CONFIG_KVM=y +CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-" CONFIG_ARM64_CRYPTO=y CONFIG_CRYPTO_SHA1_ARM64_CE=y CONFIG_CRYPTO_SHA2_ARM64_CE=ySignature · 1 line
2.20.1
Mark Rutland
reviewerRe: [PATCH v5 19/23] arm64: Enable compat vDSO support
Quoted 26 lines · Mark Rutland
Quoted 5 lines · Vincenzo Frascino
Enable compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-. For this to take effect and for the compat vDSO to compile correctly, it is required that this compiler is in the PATH.
I do not have such a compiler in my path by default.
I hope that we handle that gracefully, rather than causing a build failure.
Can't we detect the presence of the compat compiler at config time, as happens for the native compiler?
Thanks,
Mark.Quoted 8 lines
Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Signed-off-by Vincenzo Frascino <[email protected]>--- arch/arm64/configs/defconfig | 1 + 1 file changed, 1 insertion(+)
Quoted 9 lines
arch/arm64/configs/defconfig CONFIG_ACPI_APEI_MEMORY_FAILURE=y · line 123
Quoted 9 lines
arch/arm64/configs/defconfig +1
-0@@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y CONFIG_ACPI_APEI_EINJ=y CONFIG_VIRTUALIZATION=y CONFIG_KVM=y +CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
Also, the kernel.org crosstool compilers are arm-linux-gnueabi-.
Could we please avoid having a default, in the same way that we don't have a default for CROSS_COMPILE?
Otherwise, it's always gongi to be wrong for someone, and we'll never be able to change it anyway.
Thanks,
Mark.
Quoted 6 lines
Quoted 6 lines
arch/arm64/configs/defconfig +0
-0CONFIG_ARM64_CRYPTO=y CONFIG_CRYPTO_SHA1_ARM64_CE=y CONFIG_CRYPTO_SHA2_ARM64_CE=ySignature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 20/23 arm: Add support for generic vDSO
In reply to Vincenzo FrascinoThe arm vDSO library requires some adaptations to use to take advantage of the newly introduced generic vDSO library.
Introduce the following changes:
- Modification vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of elf note
--- arch/arm/Kconfig | 3 + arch/arm/include/asm/vdso/gettimeofday.h | 82 +++++++ arch/arm/include/asm/vdso_datapage.h | 25 +-- arch/arm/kernel/vdso.c | 108 ++++++--- arch/arm/vdso/Makefile | 13 +- arch/arm/vdso/note.c | 15 ++ arch/arm/vdso/vdso.lds.S | 1 + arch/arm/vdso/vgettimeofday.c | 267 ++--------------------- 8 files changed, 205 insertions(+), 309 deletions(-) create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h create mode 100644 arch/arm/vdso/note.c
arch/arm/Kconfig +3 -0
@@ -48,6 +48,8 @@ config ARM select GENERIC_SMP_IDLE_THREAD select GENERIC_STRNCPY_FROM_USER select GENERIC_STRNLEN_USER + select GENERIC_GETTIMEOFDAY + select GENERIC_VDSO_32 select HANDLE_DOMAIN_IRQ select HARDIRQS_SW_RESEND select HAVE_ARCH_AUDITSYSCALL if AEABI && !OABI_COMPAT
@@ -97,6 +99,7 @@ config ARM select HAVE_SYSCALL_TRACEPOINTS select HAVE_UID16 select HAVE_VIRT_CPU_ACCOUNTING_GEN + select HAVE_GENERIC_VDSO select IRQ_FORCED_THREADING select MODULES_USE_ELF_REL select NEED_DMA_MAP_STATE
arch/arm/include/asm/vdso/gettimeofday.h new file +82 -0
82 lines not on this page · raw message
arch/arm/include/asm/vdso_datapage.h +1 -24
@@ -22,32 +22,9 @@ #ifndef __ASSEMBLY__ +#include <vdso/datapage.h> #include <asm/page.h> -/* Try to be cache-friendly on systems that don't implement the - * generic timer: fit the unconditionally updated fields in the first - * 32 bytes. - */ -struct vdso_data { - u32 seq_count; /* sequence count - odd during updates */ - u16 tk_is_cntvct; /* fall back to syscall if false */ - u16 cs_shift; /* clocksource shift */ - u32 xtime_coarse_sec; /* coarse time */ - u32 xtime_coarse_nsec; - - u32 wtm_clock_sec; /* wall to monotonic offset */ - u32 wtm_clock_nsec; - u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */ - u32 cs_mult; /* clocksource multiplier */ - - u64 cs_cycle_last; /* last cycle value */ - u64 cs_mask; /* clocksource mask */ - - u64 xtime_clock_snsec; /* CLOCK_REALTIME sub-ns base */ - u32 tz_minuteswest; /* timezone info for gettimeofday(2) */ - u32 tz_dsttime; -}; - union vdso_data_store { struct vdso_data data; u8 page[PAGE_SIZE];
arch/arm/kernel/vdso.c +78 -30
139 lines not on this page · raw message
arch/arm/vdso/Makefile +12 -1
@@ -1,7 +1,13 @@ # SPDX-License-Identifier: GPL-2.0 + +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before +# the inclusion of generic Makefile. +ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32 +include $(srctree)/lib/vdso/Makefile + hostprogs-y := vdsomunge -obj-vdso := vgettimeofday.o datapage.o +obj-vdso := vgettimeofday.o datapage.o note.o # Build rules targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.so.raw vdso.lds
@@ -25,7 +31,11 @@ CFLAGS_REMOVE_vdso.o = -pg # Force -O2 to avoid libgcc dependencies CFLAGS_REMOVE_vgettimeofday.o = -pg -Os +ifeq ($(c-gettimeofday-y),) CFLAGS_vgettimeofday.o = -O2 +else +CFLAGS_vgettimeofday.o = -O2 -include $(c-gettimeofday-y) +endif # Disable gcov profiling for VDSO code GCOV_PROFILE := n
@@ -39,6 +49,7 @@ $(obj)/vdso.o : $(obj)/vdso.so # Link rule for the .so file $(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE $(call if_changed,vdsold) + $(call if_changed,vdso_check) $(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/vdsomunge FORCE $(call if_changed,vdsomunge)
arch/arm/vdso/note.c new file +15 -0
@@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (C) 2012-2018 ARM Limited +* +* This supplies .note.* sections to go into the PT_NOTE inside the vDSO text. +* Here we can supply some information useful to userland. +*/ + +#include <linux/uts.h> +#include <linux/version.h> +#include <linux/elfnote.h> +#include <linux/build-salt.h> + +ELFNOTE32 ("Linux" ,0 ,LINUX_VERSION_CODE ); +BUILD_SALT ;
arch/arm/vdso/vdso.lds.S +1 -0
@@ -82,6 +82,7 @@ VERSION global: __vdso_clock_gettime; __vdso_gettimeofday; + __vdso_clock_getres; local: *; }; }
arch/arm/vdso/vgettimeofday.c +13 -254
283 lines not on this page · raw message
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 21/23 mips: Add support for generic vDSO
In reply to Vincenzo FrascinoThe mips vDSO library requires some adaptations to take advantage of the newly introduced generic vDSO library.
Introduce the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
---
arch/mips/Kconfig | 2 +
arch/mips/include/asm/vdso.h | 60 +----
arch/mips/include/asm/vdso/gettimeofday.h | 165 +++++++++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 107 +++++++--
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +++
11 files changed, 313 insertions(+), 356 deletions(-)
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.carch/mips/Kconfig +2 -0
@@ -34,6 +34,7 @@ config MIPS select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC select GENERIC_SMP_IDLE_THREAD select GENERIC_TIME_VSYSCALL + select GENERIC_GETTIMEOFDAY select HANDLE_DOMAIN_IRQ select HAVE_ARCH_COMPILER_H select HAVE_ARCH_JUMP_LABEL
@@ -74,6 +75,7 @@ config MIPS select HAVE_STACKPROTECTOR select HAVE_SYSCALL_TRACEPOINTS select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP + select HAVE_GENERIC_VDSO select IRQ_FORCED_THREADING select ISA if EISA select MODULES_USE_ELF_RELA if MODULES && 64BIT
arch/mips/include/asm/vdso.h +4 -56
83 lines not on this page · raw message
arch/mips/include/asm/vdso/gettimeofday.h new file +165 -0
165 lines not on this page · raw message
arch/mips/include/asm/vdso/vdso.h renamed from arch/mips/vdso/vdso.h +3 -3
@@ -72,14 +72,14 @@ static inline unsigned long get_vdso_base(void)return addr; } -static inline const union mips_vdso_data *get_vdso_data (void ) +static inline const struct vdso_data *get_vdso_data (void ) { -return (const union mips_vdso_data *)(get_vdso_base () -PAGE_SIZE ); +return (const struct vdso_data *)(get_vdso_base () -PAGE_SIZE ); }#ifdef CONFIG_CLKSRC_MIPS_GIC -static inline void __iomem *get_gic (const union mips_vdso_data *data) +static inline void __iomem *get_gic (const struct vdso_data *data) {return (void __iomem *)data -PAGE_SIZE ; }
arch/mips/kernel/vdso.c +87 -20
139 lines not on this page · raw message
arch/mips/vdso/Makefile +21 -2
@@ -1,6 +1,12 @@ # SPDX-License-Identifier: GPL-2.0 # Objects to go into the VDSO. -obj-vdso-y := elf.o gettimeofday.o sigreturn.o + +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before +# the inclusion of generic Makefile. +ARCH_REL_TYPE_ABS := R_MIPS_JUMP_SLOT|R_MIPS_GLOB_DAT +include $(srctree)/lib/vdso/Makefile + +obj-vdso-y := elf.o vgettimeofday.o sigreturn.o # Common compiler flags between ABIs. ccflags-vdso := \
@@ -15,15 +21,23 @@ ifdef CONFIG_CC_IS_CLANG ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS)) endif +# +# The -fno-jump-tables flag only prevents the compiler from generating +# jump tables but does not prevent the compiler from emitting absolute +# offsets. cflags-vdso := $(ccflags-vdso) \ $(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \ -O2 -g -fPIC -fno-strict-aliasing -fno-common -fno-builtin -G 0 \ - -DDISABLE_BRANCH_PROFILING \ + -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \ $(call cc-option, -fno-asynchronous-unwind-tables) \ $(call cc-option, -fno-stack-protector) aflags-vdso := $(ccflags-vdso) \ -D__ASSEMBLY__ -Wa,-gdwarf-2 +ifneq ($(c-gettimeofday-y),) +CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y) +endif + # # For the pre-R6 code in arch/mips/vdso/vdso.h for locating # the base address of VDSO, the linker will emit a R_MIPS_PC32
@@ -50,6 +64,8 @@ VDSO_LDFLAGS := \ $(call cc-ldoption, -Wl$(comma)--hash-style=sysv) \ $(call cc-ldoption, -Wl$(comma)--build-id) +CFLAGS_REMOVE_vdso.o = -pg + GCOV_PROFILE := n UBSAN_SANITIZE := n
@@ -98,6 +114,7 @@ $(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi) $(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE $(call if_changed,vdsold) + $(call if_changed,vdso_check) $(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \ $(obj)/genvdso FORCE
@@ -136,6 +153,7 @@ $(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE $(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE $(call if_changed,vdsold) + $(call if_changed,vdso_check) $(obj)/vdso-o32-image.c: VDSO_NAME := o32 $(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
@@ -176,6 +194,7 @@ $(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE $(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE $(call if_changed,vdsold) + $(call if_changed,vdso_check) $(obj)/vdso-n32-image.c: VDSO_NAME := n32 $(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
arch/mips/vdso/elf.S +1 -1
@@ -8,7 +8,7 @@ * option) any later version. */ -#include "vdso.h" +#include <asm/vdso/vdso.h> #include <asm/isa-rev.h>
arch/mips/vdso/gettimeofday.c deleted +0 -273
273 lines not on this page · raw message
arch/mips/vdso/sigreturn.S +1 -1
@@ -8,7 +8,7 @@ * option) any later version. */ -#include "vdso.h" +#include <asm/vdso/vdso.h> #include <uapi/asm/unistd.h>
arch/mips/vdso/vdso.lds.S +1 -0
@@ -99,6 +99,7 @@ VERSION global: __vdso_clock_gettime; __vdso_gettimeofday; + __vdso_clock_getres; #endif local: *; };
arch/mips/vdso/vgettimeofday.c new file +28 -0
@@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* ARM64 compat userspace implementations of gettimeofday() and similar. +* +* Copyright (C) 2018 ARM Limited +* +*/ +#include <linux/time.h> +#include <linux/types.h> + +notrace int __vdso_clock_gettime(clockid_t clock, +struct __vdso_timespec *ts) +{ +return __cvdso_clock_gettime (clock, ts); +} + +notrace int __vdso_gettimeofday(struct __vdso_timeval *tv, +struct timezone *tz) +{ +return __cvdso_gettimeofday (tv, tz); +} + +notrace int __vdso_clock_getres(clockid_t clock_id, +struct __vdso_timespec *res) +{ +return __cvdso_clock_getres (clock_id, res); +} +
Signature · 1 line
2.20.1
Vincenzo Frascino
maintainerPATCH v5 22/23 x86: Add support for generic vDSO
In reply to Vincenzo FrascinoThe x86 vDSO library requires some adaptations to take advantage of the newly introduced generic vDSO library.
Introduce the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
--- arch/x86/Kconfig | 3 + arch/x86/entry/vdso/Makefile | 9 + arch/x86/entry/vdso/vclock_gettime.c | 239 +++-------------------- arch/x86/entry/vdso/vdso.lds.S | 2 + arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 + arch/x86/entry/vdso/vdsox32.lds.S | 1 + arch/x86/entry/vsyscall/vsyscall_gtod.c | 110 +++++++---- arch/x86/include/asm/vdso/gettimeofday.h | 203 +++++++++++++++++++ arch/x86/include/asm/vgtod.h | 73 +------ arch/x86/include/asm/vvar.h | 2 +- 10 files changed, 318 insertions(+), 325 deletions(-) create mode 100644 arch/x86/include/asm/vdso/gettimeofday.h
arch/x86/Kconfig +3 -0
@@ -18,6 +18,7 @@ config X86_32 select HAVE_GENERIC_DMA_COHERENT select MODULES_USE_ELF_REL select OLD_SIGACTION + select GENERIC_VDSO_32 config X86_64 def_bool y
@@ -111,6 +112,7 @@ config X86 select GENERIC_STRNCPY_FROM_USER select GENERIC_STRNLEN_USER select GENERIC_TIME_VSYSCALL + select GENERIC_GETTIMEOFDAY select HARDLOCKUP_CHECK_TIMESTAMP if X86_64 select HAVE_ACPI_APEI if ACPI select HAVE_ACPI_APEI_NMI if ACPI
@@ -194,6 +196,7 @@ config X86 select HAVE_SYSCALL_TRACEPOINTS select HAVE_UNSTABLE_SCHED_CLOCK select HAVE_USER_RETURN_NOTIFIER + select HAVE_GENERIC_VDSO select HOTPLUG_SMT if SMP select IRQ_FORCED_THREADING select NEED_SG_DMA_LENGTH
arch/x86/entry/vdso/Makefile +9 -0
@@ -3,6 +3,12 @@ # Building vDSO images for x86. # +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before +# the inclusion of generic Makefile. +ARCH_REL_TYPE_ABS := R_X86_64_JUMP_SLOT|R_X86_64_GLOB_DAT|R_X86_64_RELATIVE| +ARCH_REL_TYPE_ABS += R_386_GLOB_DAT|R_386_JMP_SLOT|R_386_RELATIVE +include $(srctree)/lib/vdso/Makefile + KBUILD_CFLAGS += $(DISABLE_LTO) KASAN_SANITIZE := n UBSAN_SANITIZE := n
@@ -51,6 +57,7 @@ VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 --no-undefined \ $(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE $(call if_changed,vdso) + $(call if_changed,vdso_check) HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi -I$(srctree)/arch/$(SUBARCH)/include/uapi hostprogs-y += vdso2c
@@ -121,6 +128,7 @@ $(obj)/%.so: $(obj)/%.so.dbg $(obj)/vdsox32.so.dbg: $(obj)/vdsox32.lds $(vobjx32s) FORCE $(call if_changed,vdso) + $(call if_changed,vdso_check) CPPFLAGS_vdso32.lds = $(CPPFLAGS_vdso.lds) VDSO_LDFLAGS_vdso32.lds = -m elf_i386 -soname linux-gate.so.1
@@ -160,6 +168,7 @@ $(obj)/vdso32.so.dbg: FORCE \ $(obj)/vdso32/system_call.o \ $(obj)/vdso32/sigreturn.o $(call if_changed,vdso) + $(call if_changed,vdso_check) # # The DSO images are built using a special linker script.
arch/x86/entry/vdso/vclock_gettime.c +24 -215
264 lines not on this page · raw message
arch/x86/entry/vdso/vdso.lds.S +2 -0
@@ -25,6 +25,8 @@ VERSION { __vdso_getcpu; time; __vdso_time; + clock_getres; + __vdso_clock_getres; local: *; }; }
arch/x86/entry/vdso/vdso32/vdso32.lds.S +1 -0
@@ -26,6 +26,7 @@ VERSION __vdso_clock_gettime; __vdso_gettimeofday; __vdso_time; + __vdso_clock_getres; }; LINUX_2.5 {
arch/x86/entry/vdso/vdsox32.lds.S +1 -0
@@ -21,6 +21,7 @@ VERSION { __vdso_gettimeofday; __vdso_getcpu; __vdso_time; + __vdso_clock_getres; local: *; }; }
arch/x86/entry/vsyscall/vsyscall_gtod.c +71 -39
136 lines not on this page · raw message
arch/x86/include/asm/vdso/gettimeofday.h new file +203 -0
203 lines not on this page · raw message
arch/x86/include/asm/vgtod.h +3 -70
89 lines not on this page · raw message
arch/x86/include/asm/vvar.h +1 -1
@@ -44,7 +44,7 @@ extern char __vvar_page;/* DECLARE_VVAR(offset, type, name) */ -DECLARE_VVAR(128, struct vsyscall_gtod_data, vsyscall_gtod_data) +DECLARE_VVAR(128, struct vdso_data, vdso_data)#undef DECLARE_VVAR
Signature · 1 line
2.20.1
Re: [PATCH v5 22/23] x86: Add support for generic vDSO
Quoted 13 lines · Vincenzo Frascino
void
update_vsyscall_tz(void)
{arch/x86/entry/vsyscall/vsyscall_gtod.c +2
-4- vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest; - vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime; + vdso_data.tz_minuteswest = sys_tz.tz_minuteswest; + vdso_data.tz_dsttime = sys_tz.tz_dsttime; }void update_vsyscall (struct timekeeper *tk) {int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode; -struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data; -struct vgtod_ts *base;
arch/x86/entry/vsyscall/vsyscall_gtod.c line 41
Quoted 10 lines
arch/x86/entry/vsyscall/vsyscall_gtod.c +3
-1+struct vdso_data *vdata = &vdso_data; +struct vdso_timestamp *vdso_ts;u64 nsec;/* Mark the new vclock used. */ BUILD_BUG_ON (VCLOCK_MAX >=32 );WRITE_ONCE (vclocks_used,READ_ONCE (vclocks_used) | (1 << vclock_mode)); -gtod_write_begin (vdata); +vdso_write_begin (vdata);
This begs the question why this update code is still architecture specific. The vdso data is now generic, so the whole update can be generic as well. The only x86 specific thing is this perhaps:
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
vdata->clock_mode = vclock_mode;
But either than can be made generic and mandatory or easily resolved with a trivial arch_update_vsyscall() inline.
Thanks,
tglx
Vincenzo Frascino
maintainerRe: [PATCH v5 22/23] x86: Add support for generic vDSO
Quoted 29 lines · Thomas Gleixner
Quoted 13 lines · Vincenzo Frascino
void
update_vsyscall_tz(void)
{arch/x86/entry/vsyscall/vsyscall_gtod.c +2
-4- vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest; - vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime; + vdso_data.tz_minuteswest = sys_tz.tz_minuteswest; + vdso_data.tz_dsttime = sys_tz.tz_dsttime; }void update_vsyscall (struct timekeeper *tk) {int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode; -struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data; -struct vgtod_ts *base;arch/x86/entry/vsyscall/vsyscall_gtod.c line 41
Quoted 10 lines
arch/x86/entry/vsyscall/vsyscall_gtod.c +3
-1+struct vdso_data *vdata = &vdso_data; +struct vdso_timestamp *vdso_ts;u64 nsec;/* Mark the new vclock used. */ BUILD_BUG_ON (VCLOCK_MAX >=32 );WRITE_ONCE (vclocks_used,READ_ONCE (vclocks_used) | (1 << vclock_mode)); -gtod_write_begin (vdata); +vdso_write_begin (vdata);This begs the question why this update code is still architecture specific. The vdso data is now generic, so the whole update can be generic as well. The only x86 specific thing is this perhaps:
Quoted 7 lines
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;vdata->clock_mode = vclock_mode;But either than can be made generic and mandatory or easily resolved with a trivial
arch_update_vsyscall()inline.
I agree, I will make it generic in the next patch series.
Quoted 4 lines
Thanks,
tglx
Signature · 2 lines
Regards, Vincenzo
Vincenzo Frascino
maintainerPATCH v5 23/23 kselftest: Extend vDSO selftest
In reply to Vincenzo FrascinoThe current version of the multiarch vDSO selftest verifies only gettimeofday.
Extend the vDSO selftest to the other library functions:
- time
- clock_getres
- clock_gettime
The extension has been used to verify the unified vdso library on the supported architectures.
--- tools/testing/selftests/vDSO/Makefile | 2 + tools/testing/selftests/vDSO/vdso_full_test.c | 246 ++++++++++++++++++ 2 files changed, 248 insertions(+) create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.c
tools/testing/selftests/vDSO/Makefile +2 -0
@@ -5,6 +5,7 @@ uname_M := $(shell uname -m 2>/dev/null || echo not) ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/x86/ -e s/x86_64/x86/) TEST_GEN_PROGS := $(OUTPUT)/vdso_test +TEST_GEN_PROGS += $(OUTPUT)/vdso_full_test ifeq ($(ARCH),x86) TEST_GEN_PROGS += $(OUTPUT)/vdso_standalone_test_x86 endif
@@ -18,6 +19,7 @@ endif all: $(TEST_GEN_PROGS) $(OUTPUT)/vdso_test: parse_vdso.c vdso_test.c +$(OUTPUT)/vdso_full_test: parse_vdso.c vdso_full_test.c $(OUTPUT)/vdso_standalone_test_x86: vdso_standalone_test_x86.c parse_vdso.c $(CC) $(CFLAGS) $(CFLAGS_vdso_standalone_test_x86) \ vdso_standalone_test_x86.c parse_vdso.c \
tools/testing/selftests/vDSO/vdso_full_test.c new file +246 -0
246 lines not on this page · raw message
Signature · 1 line
2.20.1
Quoted 9 lines · Vincenzo Frascino
This implementation contains the following design choices:
* Every architecture defines the arch specific code in an header in "asm/vdso/".
* The generic implementation includes the arch specific one and lives in "lib/vdso".
* The arch specific code for gettimeofday lives in
"<arch path>/vdso/gettimeofday.c" and includes the generic code only.
* This approach allows to consolidate the common code in a single place with the benefit of avoiding code duplication.
This is moving into the right direction. Good job!
Thanks,
tglx
Arnd Bergmann
maintainerQuoted 8 lines · Vincenzo Frascino
vDSO (virtual dynamic shared object) is a mechanism that the Linux kernel provides as an alternative to system calls to reduce where possible the costs in terms of cycles.
This is possible because certain syscalls likegettimeofday()do not write any data and return one or more values that are stored in the kernel, which makes relatively safe calling them directly as a library function.
I tried applying the series on top of v5.0-rc7, and got a set of build warnings with arm64 defconfig:
In file included from /git/arm-soc/arch/arm64/include/asm/thread_info.h:30:0,
from /git/arm-soc/include/linux/thread_info.h:38,
from /git/arm-soc/arch/arm64/include/asm/preempt.h:5,
from /git/arm-soc/include/linux/preempt.h:78,
from /git/arm-soc/include/linux/spinlock.h:51,
from /git/arm-soc/include/linux/seqlock.h:36,
from /git/arm-soc/include/linux/time.h:6,
from /git/arm-soc/lib/vdso/gettimeofday.c:7,
from <command-line>:0:/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
/git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left shift count >= width of type [enabled by default]
return kimage_vaddr - KIMAGE_VADDR;
^
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left shift count >= width of type [enabled by default]
return __virt_to_phys((unsigned long)(x));
^
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left shift count >= width of type [enabled by default] /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
/git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left shift count >= width of type [enabled by default]
return (void *)(__phys_to_virt(x));
^
In file included from /git/arm-soc/arch/arm64/include/asm/pgtable-hwdef.h:19:0,
from /git/arm-soc/arch/arm64/include/asm/processor.h:46,
from /git/arm-soc/arch/arm64/include/asm/elf.h:129,
from /git/arm-soc/include/linux/elf.h:5,
from /git/arm-soc/include/linux/elfnote.h:62,
from /git/arm-soc/arch/arm64/kernel/vdso32/note.c:11:/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
/git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left shift count >= width of type [enabled by default]
return kimage_vaddr - KIMAGE_VADDR;
^
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left shift count >= width of type [enabled by default]
return __virt_to_phys((unsigned long)(x));
^
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left shift count >= width of type [enabled by default] /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
/git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left shift count >= width of type [enabled by default]
return (void *)(__phys_to_virt(x));
^
I think this is all harmless, but we need to limit the set of header files that can be included indirectly when compiling the vdso in arm32 mode.
Arnd
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Vincenzo Frascino
maintainerHi Arnd,
thank you for testing my code.
Quoted 64 lines · Arnd Bergmann
Quoted 8 lines · Vincenzo Frascino
vDSO (virtual dynamic shared object) is a mechanism that the Linux kernel provides as an alternative to system calls to reduce where possible the costs in terms of cycles.
This is possible because certain syscalls likegettimeofday()do not write any data and return one or more values that are stored in the kernel, which makes relatively safe calling them directly as a library function.I tried applying the series on top of v5.0-rc7, and got a set of build warnings with arm64 defconfig:
In file included from /git/arm-soc/arch/arm64/include/asm/thread_info.h:30:0,
from /git/arm-soc/include/linux/thread_info.h:38, from /git/arm-soc/arch/arm64/include/asm/preempt.h:5, from /git/arm-soc/include/linux/preempt.h:78, from /git/arm-soc/include/linux/spinlock.h:51, from /git/arm-soc/include/linux/seqlock.h:36, from /git/arm-soc/include/linux/time.h:6, from /git/arm-soc/lib/vdso/gettimeofday.c:7, from <command-line>:0:/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
/git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left shift count >= width of type [enabled by default]
returnkimage_vaddr-KIMAGE_VADDR;
^
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left shift count >= width of type [enabled by default]
return__virt_to_phys((unsigned long)(x));
^
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left shift count >= width of type [enabled by default] /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
/git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left shift count >= width of type [enabled by default]
return (void *)(__phys_to_virt(x));
^
In file included from /git/arm-soc/arch/arm64/include/asm/pgtable-hwdef.h:19:0,from /git/arm-soc/arch/arm64/include/asm/processor.h:46, from /git/arm-soc/arch/arm64/include/asm/elf.h:129, from /git/arm-soc/include/linux/elf.h:5, from /git/arm-soc/include/linux/elfnote.h:62, from /git/arm-soc/arch/arm64/kernel/vdso32/note.c:11:/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
/git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left shift count >= width of type [enabled by default]
returnkimage_vaddr-KIMAGE_VADDR;
^
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left shift count >= width of type [enabled by default]
return__virt_to_phys((unsigned long)(x));
^
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left shift count >= width of type [enabled by default] /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
/git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left shift count >= width of type [enabled by default]
return (void *)(__phys_to_virt(x));
^
Quoted 3 lines
I think this is all harmless, but we need to limit the set of header files that can be included indirectly when compiling the vdso in arm32 mode.
I agree with you on the principle, but I am unable to replicate the warnings. Did you make any customization to the default defconfig? What is the top commit-id you have on your tree?
Quoted 2 lines
Arnd
Signature · 7 lines
Regards, Vincenzo _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Arnd Bergmann
maintainerQuoted 8 lines · Vincenzo Frascino
thank you for testing my code.
Quoted 4 lines · Arnd Bergmann
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
<[email protected]> wrote:
^
Quoted 7 lines
Quoted 3 lines
I think this is all harmless, but we need to limit the set of header files that can be included indirectly when compiling the vdso in arm32 mode.
I agree with you on the principle, but I am unable to replicate the warnings. Did you make any customization to the default defconfig? What is the top commit-id you have on your tree?
I observed this on top of v5.0-rc7 earlier, and have since rebased to today's linux-next, where it still happens with the regular 'make defconfig', on all compiler versions I have (4.8 through 8.1).
Arnd
Vincenzo Frascino
maintainerQuoted 14 lines · Arnd Bergmann
Quoted 12 lines · Vincenzo Frascino
thank you for testing my code.
Quoted 7 lines · Arnd Bergmann
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
<[email protected]> wrote:
^I think this is all harmless, but we need to limit the set of header files that can be included indirectly when compiling the vdso in arm32 mode.
Quoted 8 lines
Quoted 3 lines
I agree with you on the principle, but I am unable to replicate the warnings. Did you make any customization to the default defconfig? What is the top commit-id you have on your tree?
I observed this on top of v5.0-rc7 earlier, and have since rebased to today's linux-next, where it still happens with the regular 'make defconfig', on all compiler versions I have (4.8 through 8.1).
Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
Quoted 2 lines
Arnd
Signature · 2 lines
Regards, Vincenzo
Arnd Bergmann
maintainerQuoted 20 lines · Vincenzo Frascino
Quoted 18 lines · Arnd Bergmann
Quoted 15 lines · Vincenzo Frascino
thank you for testing my code.
Quoted 7 lines · Arnd Bergmann
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
<[email protected]> wrote:
^I think this is all harmless, but we need to limit the set of header files that can be included indirectly when compiling the vdso in arm32 mode.
I agree with you on the principle, but I am unable to replicate the warnings. Did you make any customization to the default defconfig? What is the top commit-id you have on your tree?
Quoted 6 lines
Quoted 4 lines
I observed this on top of v5.0-rc7 earlier, and have since rebased to today's linux-next, where it still happens with the regular 'make defconfig', on all compiler versions I have (4.8 through 8.1).
Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
I just found a related issue: when CONFIG_CPU_BIG_ENDIAN is set, everything breaks in the compat vdso because of the included arm64 headers in arm32 code. That should be easier to reproduce. For the moment, I'm using this patch:
arch/arm64/Kconfig +1 -1
@@ -103,7 +103,7 @@ config ARM64 select GENERIC_STRNLEN_USER select GENERIC_TIME_VSYSCALL select GENERIC_GETTIMEOFDAY - select GENERIC_COMPAT_VDSO + select GENERIC_COMPAT_VDSO if !CPU_BIG_ENDIAN select HANDLE_DOMAIN_IRQ select HARDIRQS_SW_RESEND select HAVE_PCI
Arnd
Vincenzo Frascino
maintainerHi Arnd,
Quoted 39 lines · Arnd Bergmann
Quoted 20 lines · Vincenzo Frascino
Quoted 18 lines · Arnd Bergmann
Quoted 15 lines · Vincenzo Frascino
thank you for testing my code.
Quoted 7 lines · Arnd Bergmann
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
<[email protected]> wrote:
^I think this is all harmless, but we need to limit the set of header files that can be included indirectly when compiling the vdso in arm32 mode.
I agree with you on the principle, but I am unable to replicate the warnings. Did you make any customization to the default defconfig? What is the top commit-id you have on your tree?
Quoted 6 lines
Quoted 4 lines
I observed this on top of v5.0-rc7 earlier, and have since rebased to today's linux-next, where it still happens with the regular 'make defconfig', on all compiler versions I have (4.8 through 8.1).
Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
I just found a related issue: when
CONFIG_CPU_BIG_ENDIANis set, everything breaks in the compat vdso because of the included arm64 headers in arm32 code. That should be easier to reproduce. For the moment, I'm using this patch:
arch/arm64/Kconfig config ARM64 · line 109
Quoted 8 lines
arch/arm64/Kconfig +1
-1@@ -103,7 +103,7 @@ config ARM64 select GENERIC_STRNLEN_USER select GENERIC_TIME_VSYSCALL select GENERIC_GETTIMEOFDAY - select GENERIC_COMPAT_VDSO + select GENERIC_COMPAT_VDSO if !CPU_BIG_ENDIAN select HANDLE_DOMAIN_IRQ select HARDIRQS_SW_RESEND select HAVE_PCI
Thanks for this, but I can't still reproduce the issue.
I rebased my patches on:
commit 42fd8df9d1d9c83046226e771a4f5f3d05a706ff (tag: next-20190228, linux-next/master)
and enabled CONFIG_CPU_BIG_ENDIAN.
Quoted 2 lines
Arnd
Signature · 2 lines
Regards, Vincenzo