glibc: Perform rseq(2) registration at C startup and thread creation (v6)
RFC PATCH 1/4 glibc: Perform rseq(2) registration at C startup and thread creation (v6)
Register rseq(2) TLS for each thread (including main), and unregister for each thread (excluding main). "rseq" stands for Restartable Sequences.
See the rseq(2) man page proposed here:
https://lkml.org/lkml/2018/9/19/647
This patch is based on glibc commit a502c5294. The rseq(2) system call was merged into Linux 4.18.
---
Changes since v1:
- Move __rseq_refcount to an extra field at the end of __rseq_abi to eliminate one symbol.
All libraries/programs which try to register rseq (glibc, early-adopter applications, early-adopter libraries) should use the rseq refcount. It becomes part of the ABI within a user-space process, but it's not part of the ABI shared with the kernel per se.
- Restructure how this code is organized so glibc keeps building on non-Linux targets.
- Use non-weak symbol for __rseq_abi.
- Move rseq registration/unregistration implementation into its own nptl/rseq.c compile unit.
- Move __rseq_abi symbol under GLIBC_2.29.
Changes since v2:
- Move __rseq_refcount to its own symbol, which is less ugly than trying to play tricks with the rseq uapi.
- Move __rseq_abi from nptl to csu (C start up), so it can be used across glibc, including memory allocator and sched_getcpu(). The __rseq_refcount symbol is kept in nptl, because there is no reason to use it elsewhere in glibc.
Changes since v3:
- Set __rseq_refcount TLS to 1 on register/set to 0 on unregister because glibc is the first/last user.
- Unconditionally register/unregister rseq at thread start/exit, because glibc is the first/last user.
- Add missing abilist items.
- Rebase on glibc master commit a502c5294.
- Add NEWS entry.
Changes since v4:
- Do not use "weak" symbols for __rseq_abi and __rseq_refcount. Based on "System V Application Binary Interface", weak only affects the link editor, not the dynamic linker.
- Install a new sys/rseq.h system header on Linux, which contains the RSEQ_SIG definition, __rseq_abi declaration and __rseq_refcount declaration. Move those definition/declarations from rseq-internal.h to the installed sys/rseq.h header.
- Considering that rseq is only available on Linux, move csu/rseq.c to sysdeps/unix/sysv/linux/rseq-sym.c.
- Move __rseq_refcount from nptl/rseq.c to sysdeps/unix/sysv/linux/rseq-sym.c, so it is only defined on Linux.
- Move both ABI definitions for __rseq_abi and __rseq_refcount to sysdeps/unix/sysv/linux/Versions, so they only appear on Linux.
- Document __rseq_abi and __rseq_refcount volatile.
- Document the RSEQ_SIG signature define.
- Move registration functions from rseq.c to rseq-internal.h static inline functions. Introduce empty stubs in misc/rseq-internal.h, which can be overridden by architecture code in sysdeps/unix/sysv/linux/rseq-internal.h.
- Rename __rseq_register_current_thread and __rseq_unregister_current_thread to rseq_register_current_thread and rseq_unregister_current_thread, now that those are only visible as internal static inline functions.
- Invoke rseq_register_current_thread() from libc-start.c LIBC_START_MAIN rather than nptl init, so applications not linked against libpthread.so have rseq registered for their main() thread. Note that it is invoked separately for SHARED and !SHARED builds.
Changes since v5:
- Replace __rseq_refcount by __rseq_lib_abi, which contains two
uint32_t: register_state and refcount. The "register_state" field allows inhibiting rseq registration from signal handlers nested on top of glibc registration and occuring after rseq unregistration by glibc.
- Introduce enum rseq_register_state, which contains the states allowed for the struct rseq_lib_abi register_state field.
--- NEWS | 6 ++ csu/libc-start.c | 12 ++- misc/Makefile | 3 +- misc/rseq-internal.h | 34 +++++++ nptl/pthread_create.c | 9 ++ sysdeps/unix/sysv/linux/Makefile | 5 +- sysdeps/unix/sysv/linux/Versions | 4 + sysdeps/unix/sysv/linux/aarch64/libc.abilist | 2 + sysdeps/unix/sysv/linux/alpha/libc.abilist | 2 + sysdeps/unix/sysv/linux/arm/libc.abilist | 2 + sysdeps/unix/sysv/linux/hppa/libc.abilist | 2 + sysdeps/unix/sysv/linux/i386/libc.abilist | 2 + sysdeps/unix/sysv/linux/ia64/libc.abilist | 2 + .../sysv/linux/m68k/coldfire/libc.abilist | 2 + .../unix/sysv/linux/m68k/m680x0/libc.abilist | 2 + .../unix/sysv/linux/microblaze/libc.abilist | 2 + .../sysv/linux/mips/mips32/fpu/libc.abilist | 2 + .../sysv/linux/mips/mips32/nofpu/libc.abilist | 2 + .../sysv/linux/mips/mips64/n32/libc.abilist | 2 + .../sysv/linux/mips/mips64/n64/libc.abilist | 2 + sysdeps/unix/sysv/linux/nios2/libc.abilist | 2 + .../linux/powerpc/powerpc32/fpu/libc.abilist | 2 + .../powerpc/powerpc32/nofpu/libc.abilist | 2 + .../linux/powerpc/powerpc64/libc-le.abilist | 2 + .../sysv/linux/powerpc/powerpc64/libc.abilist | 2 + .../unix/sysv/linux/riscv/rv64/libc.abilist | 2 + sysdeps/unix/sysv/linux/rseq-internal.h | 91 +++++++++++++++++++ sysdeps/unix/sysv/linux/rseq-sym.c | 54 +++++++++++ .../unix/sysv/linux/s390/s390-32/libc.abilist | 2 + .../unix/sysv/linux/s390/s390-64/libc.abilist | 2 + sysdeps/unix/sysv/linux/sh/libc.abilist | 2 + .../sysv/linux/sparc/sparc32/libc.abilist | 2 + .../sysv/linux/sparc/sparc64/libc.abilist | 2 + sysdeps/unix/sysv/linux/sys/rseq.h | 64 +++++++++++++ .../unix/sysv/linux/x86_64/64/libc.abilist | 2 + .../unix/sysv/linux/x86_64/x32/libc.abilist | 2 + 36 files changed, 328 insertions(+), 6 deletions(-) create mode 100644 misc/rseq-internal.h create mode 100644 sysdeps/unix/sysv/linux/rseq-internal.h create mode 100644 sysdeps/unix/sysv/linux/rseq-sym.c create mode 100644 sysdeps/unix/sysv/linux/sys/rseq.h
NEWS +6 -0
@@ -35,6 +35,12 @@ Major new features: different directory. This is a GNU extension and similar to the Solaris function of the same name. +* Support for automatically registering threads with the Linux rseq(2) + system call has been added. This system call is implemented starting + from Linux 4.18. In order to be activated, it requires that glibc is built + against kernel headers that include this system call, and that glibc + detects availability of that system call at runtime. + Deprecated and removed features, and other changes affecting compatibility: * The glibc.tune tunable namespace has been renamed to glibc.cpu and the
csu/libc-start.c +9 -3
@@ -22,6 +22,7 @@#include <ldsodefs.h> #include <exit-thread.h> #include <libc-internal.h> +#include <rseq-internal.h> #include <elf/dl-tunables.h>
@@ -140,7 +141,10 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), __libc_multiple_libcs = &_dl_starting_up && !_dl_starting_up; -#ifndef SHARED +#ifdef SHARED +/* Register rseq ABI to the kernel. */ + (void )rseq_register_current_thread (); +#else _dl_relocate_static_pie ();char **ev = &argv[argc +1 ];
@@ -218,6 +222,9 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), }# endif +/* Register rseq ABI to the kernel. */ + (void )rseq_register_current_thread (); +/* Initialize libpthread if linked in. */ if (__pthread_initialize_minimal !=NULL )__pthread_initialize_minimal ();
@@ -230,8 +237,7 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL),# else __pointer_chk_guard_local = pointer_chk_guard;# endif - -#endif /* !SHARED */ +#endif /* Register the destructor of the dynamic linker if there is any. */ if (__glibc_likely (rtld_fini !=NULL ))
misc/Makefile +2 -1
@@ -36,7 +36,8 @@ headers := sys/uio.h bits/uio-ext.h bits/uio_lim.h \ syslog.h sys/syslog.h \ bits/syslog.h bits/syslog-ldbl.h bits/syslog-path.h bits/error.h \ bits/select2.h bits/hwcap.h sys/auxv.h \ - sys/sysmacros.h bits/sysmacros.h bits/types/struct_iovec.h + sys/sysmacros.h bits/sysmacros.h bits/types/struct_iovec.h \ + rseq-internal.h routines := brk sbrk sstk ioctl \ readv writev preadv preadv64 pwritev pwritev64 \
misc/rseq-internal.h new file +34 -0
@@ -0,0 +1,34 @@ +/* Copyright (C) 2018 Free Software Foundation, Inc. +This file is part of the GNU C Library. +Contributed by Mathieu Desnoyers <[email protected]>, 2018. + +The GNU C Library is free software; you can redistribute it and/or +modify it under the terms of the GNU Lesser General Public +License as published by the Free Software Foundation; either +version 2.1 of the License, or (at your option) any later version. + +The GNU C Library is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +Lesser General Public License for more details. + +You should have received a copy of the GNU Lesser General Public +License along with the GNU C Library; if not, see +<http://www.gnu.org/licenses/>. */ + +#ifndef RSEQ_INTERNAL_H +#define RSEQ_INTERNAL_H + +static inline int +rseq_register_current_thread (void ) +{ +return -1 ; +} + +static inline int +rseq_unregister_current_thread (void ) +{ +return -1 ; +} + +#endif /* rseq-internal.h */
nptl/pthread_create.c +9 -0
@@ -33,6 +33,7 @@#include <default-sched.h> #include <futex-internal.h> #include <tls-setup.h> +#include <rseq-internal.h> #include "libioP.h" #include <shlib-compat.h>
@@ -378,6 +379,7 @@ __free_tcb (struct pthread *pd) START_THREAD_DEFN {struct pthread *pd =START_THREAD_SELF ; +bool has_rseq = false;#if HP_TIMING_AVAIL /* Remember the time when the thread was started. */
@@ -396,6 +398,9 @@ START_THREAD_DEFNif (__glibc_unlikely (atomic_exchange_acq (&pd->setxid_futex,0 ) ==-2 ))futex_wake (&pd->setxid_futex,1 ,FUTEX_PRIVATE ); +/* Register rseq TLS to the kernel. */ + has_rseq = !rseq_register_current_thread (); +#ifdef __NR_set_robust_list# ifndef __ASSUME_SET_ROBUST_LISTif (__set_robust_list_avail >=0 )
@@ -573,6 +578,10 @@ START_THREAD_DEFN }#endif +/* Unregister rseq TLS from kernel. */ +if (has_rseq &&rseq_unregister_current_thread ()) +abort (); +advise_stack_range (pd->stackblock, pd->stackblock_size, (uintptr_t ) pd, pd->guardsize);
sysdeps/unix/sysv/linux/Makefile +3 -2
@@ -1,5 +1,5 @@ ifeq ($(subdir),csu) -sysdep_routines += errno-loc +sysdep_routines += errno-loc rseq-sym endif ifeq ($(subdir),assert)
@@ -43,7 +43,8 @@ sysdep_headers += sys/mount.h sys/acct.h sys/sysctl.h \ bits/siginfo-arch.h bits/siginfo-consts-arch.h \ bits/procfs.h bits/procfs-id.h bits/procfs-extra.h \ bits/procfs-prregset.h bits/mman-map-flags-generic.h \ - bits/msq-pad.h bits/sem-pad.h bits/shmlba.h bits/shm-pad.h + bits/msq-pad.h bits/sem-pad.h bits/shmlba.h bits/shm-pad.h \ + sys/rseq.h tests += tst-clone tst-clone2 tst-clone3 tst-fanotify tst-personality \ tst-quota tst-sync_file_range tst-sysconf-iov_max tst-ttyname \
sysdeps/unix/sysv/linux/Versions +4 -0
@@ -171,6 +171,10 @@ libc { mlock2; pkey_alloc; pkey_free; pkey_set; pkey_get; pkey_mprotect; } + GLIBC_2.29 { + __rseq_abi; + __rseq_lib_abi; + } GLIBC_PRIVATE { # functions used in other libraries __syscall_rt_sigqueueinfo;
sysdeps/unix/sysv/linux/aarch64/libc.abilist +2 -0
@@ -2138,4 +2138,6 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F
sysdeps/unix/sysv/linux/alpha/libc.abilist +2 -0
@@ -2033,6 +2033,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/arm/libc.abilist +2 -0
@@ -123,6 +123,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.4 _Exit F GLIBC_2.4 _IO_2_1_stderr_ D 0xa0
sysdeps/unix/sysv/linux/hppa/libc.abilist +2 -0
@@ -1880,6 +1880,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/i386/libc.abilist +2 -0
@@ -2045,6 +2045,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/ia64/libc.abilist +2 -0
@@ -1914,6 +1914,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist +2 -0
@@ -124,6 +124,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.4 _Exit F GLIBC_2.4 _IO_2_1_stderr_ D 0x98
sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist +2 -0
@@ -1989,6 +1989,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/microblaze/libc.abilist +2 -0
@@ -2130,4 +2130,6 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F
sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist +2 -0
@@ -1967,6 +1967,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist +2 -0
@@ -1965,6 +1965,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist +2 -0
@@ -1973,6 +1973,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist +2 -0
@@ -1968,6 +1968,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/nios2/libc.abilist +2 -0
@@ -2171,4 +2171,6 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F
sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist +2 -0
@@ -1993,6 +1993,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist +2 -0
@@ -1997,6 +1997,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/powerpc/powerpc64/libc-le.abilist +2 -0
@@ -2228,4 +2228,6 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F
sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist +2 -0
@@ -123,6 +123,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 _Exit F GLIBC_2.3 _IO_2_1_stderr_ D 0xe0
sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist +2 -0
@@ -2100,4 +2100,6 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F
sysdeps/unix/sysv/linux/rseq-internal.h new file +91 -0
91 lines not on this page · raw message
sysdeps/unix/sysv/linux/rseq-sym.c new file +54 -0
54 lines not on this page · raw message
sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist +2 -0
@@ -2002,6 +2002,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist +2 -0
@@ -1908,6 +1908,8 @@ GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F GLIBC_2.29 __fentry__ F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/sh/libc.abilist +2 -0
@@ -1884,6 +1884,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist +2 -0
@@ -1996,6 +1996,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist +2 -0
@@ -1937,6 +1937,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/sys/rseq.h new file +64 -0
64 lines not on this page · raw message
sysdeps/unix/sysv/linux/x86_64/64/libc.abilist +2 -0
@@ -1895,6 +1895,8 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F GLIBC_2.3 __ctype_b_loc F GLIBC_2.3 __ctype_tolower_loc F
sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist +2 -0
@@ -2146,4 +2146,6 @@ GLIBC_2.28 thrd_current F GLIBC_2.28 thrd_equal F GLIBC_2.28 thrd_sleep F GLIBC_2.28 thrd_yield F +GLIBC_2.29 __rseq_abi T 0x20 +GLIBC_2.29 __rseq_lib_abi T 0x8 GLIBC_2.29 posix_spawn_file_actions_addchdir_np F
Signature · 1 line
2.17.1
RFC PATCH 2/4 glibc: sched_getcpu(): use rseq cpu_id TLS on Linux
When available, use the cpu_id field from __rseq_abi on Linux to implement sched_getcpu(). Fall-back on the vgetcpu vDSO if unavailable.
Benchmarks:
x86-64: Intel E5-2630 [email protected], 16-core, hyperthreading
glibc sched_getcpu(): 13.7 ns (baseline) glibc sched_getcpu() using rseq: 2.5 ns (speedup: 5.5x)
inline load cpuid from __rseq_abi TLS: 0.8 ns (speedup: 17.1x)
--- sysdeps/unix/sysv/linux/sched_getcpu.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)
sysdeps/unix/sysv/linux/sched_getcpu.c +23 -2
@@ -24,8 +24,8 @@#endif #include <sysdep-vdso.h> -int -sched_getcpu (void ) +static int +vsyscall_sched_getcpu (void ) {#ifdef __NR_getcpuunsigned int cpu;
@@ -37,3 +37,24 @@ sched_getcpu (void)return -1 ;#endif } + +#ifdef __NR_rseq +#include <linux/rseq.h> + +extern __attribute__ ((tls_model ("initial-exec" ))) +__threadvolatile struct rseq __rseq_abi; + +int +sched_getcpu (void ) +{ +int cpu_id = __rseq_abi.cpu_id; + +return cpu_id >=0 ? cpu_id :vsyscall_sched_getcpu (); +} +#else +int +sched_getcpu (void ) +{ +return vsyscall_sched_getcpu (); +} +#endif
Signature · 1 line
2.17.1
----- On Jan 21, 2019, at 4:35 PM, Mathieu Desnoyers [email protected] wrote:
[...]
Quoted 7 lines
sysdeps/unix/sysv/linux/sys/rseq.h new file +0
-0@@ -0,0 +1,64 @@
[...]
sysdeps/unix/sysv/linux/sys/rseq.h line 27
Quoted 10 lines
sysdeps/unix/sysv/linux/sys/rseq.h new file +10
-0+ +#ifndef _SYS_RSEQ_H +#define _SYS_RSEQ_H 1 + +/* We use the structures declarations from the kernel headers. */ +#include <linux/rseq.h> +#include <stdint.h> + +/* Signature required before each abort handler code. */ +#define RSEQ_SIG 0x53053053
I recalled that aarch64 defines RSEQ_SIG to a different value which maps to a valid trap instruction. So I plan to move the RSEQ_SIG define to per-arch headers like this:
sysdeps/unix/sysv/linux/aarch64/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/arm/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/bits/rseq.h | 23 ++ sysdeps/unix/sysv/linux/mips/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/powerpc/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/s390/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/x86/bits/rseq.h | 24 ++
where "bits/rseq.h" contains a #error:
# error "Architecture does not define RSEQ_SIG.
sys/rseq.h will now include <bits/rseq.h>.
sysdeps/unix/sysv/linux/sys/rseq.h line 36
Quoted 9 lines
sysdeps/unix/sysv/linux/sys/rseq.h new file +9
-0+ +enum rseq_register_state +{ +/* Value RSEQ_REGISTER_ALLOWED means it is allowed to update +the refcount field and to register/unregister rseq. */ +RSEQ_REGISTER_ALLOWED =0 , +/* Value RSEQ_REGISTER_NESTED means it is temporarily forbidden +to update the refcount field or to register/unregister rseq. */ +RSEQ_REGISTER_NESTED =1 ,
I plan to rename "RSEQ_REGISTER_NESTED" to "RSEQ_REGISTER_ONGOING" which seems to better represent the current registration state.
Please let me know if anything is wrong with those changes.
Thanks,
Mathieu
Quoted 5 lines
sysdeps/unix/sysv/linux/sys/rseq.h new file +5
-0+/* Value RSEQ_REGISTER_EXITING means it is forbidden to update the +refcount field or to register/unregister rseq for the rest of the +thread's lifetime. */ + RSEQ_REGISTER_EXITING = 2, +};
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Quoted 12 lines · Mathieu Desnoyers
I recalled that aarch64 defines
RSEQ_SIGto a different value which maps to a valid trap instruction. So I plan to move theRSEQ_SIGdefine to per-arch headers like this:sysdeps/unix/sysv/linux/aarch64/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/arm/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/bits/rseq.h | 23 ++ sysdeps/unix/sysv/linux/mips/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/powerpc/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/s390/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/x86/bits/rseq.h | 24 ++
Quoted 5 lines
where "
bits/rseq.h" contains a #error:# error "Architecture does not define RSEQ_SIG.
sys/rseq.hwill now include <bits/rseq.h>.
We're trying to reduce the number of cases where most or all new glibc architecture ports need to provide a bits/ header, by making the generic headers handle the common case. So a generic header with a #error, and lots of architecture-specific headers mostly with the same value for RSEQ_SIG, seems unfortunate. I'd hope the generic header could use a generic value, with architecture-specific variants only for architectures with some reason for a different value.
Signature · 2 lines
Joseph S. Myers [email protected]
Quoted 20 lines · ----- On Jan 29, 2019, at 4:56 PM, Joseph Myers [email protected] wrote:
Quoted 12 lines · Mathieu Desnoyers
I recalled that aarch64 defines
RSEQ_SIGto a different value which maps to a valid trap instruction. So I plan to move theRSEQ_SIGdefine to per-arch headers like this:sysdeps/unix/sysv/linux/aarch64/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/arm/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/bits/rseq.h | 23 ++ sysdeps/unix/sysv/linux/mips/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/powerpc/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/s390/bits/rseq.h | 24 ++ sysdeps/unix/sysv/linux/x86/bits/rseq.h | 24 ++Quoted 5 lines
where "
bits/rseq.h" contains a #error:# error "Architecture does not define RSEQ_SIG.
sys/rseq.hwill now include <bits/rseq.h>.
Quoted 7 lines
We're trying to reduce the number of cases where most or all new glibc architecture ports need to provide a bits/ header, by making the generic headers handle the common case. So a generic header with a #error, and lots of architecture-specific headers mostly with the same value for
RSEQ_SIG, seems unfortunate. I'd hope the generic header could use a generic value, with architecture-specific variants only for architectures with some reason for a different value.
The issue here is that it would require us to decide right away what RSEQ_SIG is appropriate for all other Linux architectures supported by glibc. There are a few reasons for which an architecture can be required to specify its own RSEQ_SIG. For instance, it may need to map to an instruction defined in the instruction set, thus ensuring objdump does not get confused, and in other cases that the processor speculative execution happening just before the RSEQ_SIG really stops at the signature (hence the trap instruction on aarch64).
I'm worried that if we introduce a "default" RSEQ_SIG value for architectures currently not supported by RSEQ and we then introduce an architecture-specific signature value in the future, some applications will try to build with wrong signatures, and when the rseq system call gets eventually implemented for those architecture and a end-user upgrades his kernel, those signatures won't match between glibc rseq registration and the application rseq abort handlers, thus leading to hard-to-reproduce segmentation faults delivered by the kernel checking those signatures upon rseq abort.
This upgrade story is far from ideal.
My thinking was to put the #error in the generic header, so architectures that are not supported yet cannot build against rseq.h at all, so we don't end up in a broken upgrade scenario. I'm open to alternative ways to do it though, as long as we don't let not-yet-supported architectures build broken code.
Thoughts ?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Quoted 4 lines · Mathieu Desnoyers
My thinking was to put the #error in the generic header, so architectures that are not supported yet cannot build against
rseq.hat all, so we don't end up in a broken upgrade scenario. I'm open to alternative ways to do it though, as long as we don't let not-yet-supported architectures build broken code.
Any case with #error in installed glibc headers needs special-casing in check-installed-headers.sh (and, thus, such errors are to be discouraged).
Cases where architectures commonly need their own bits/ headers, especially where those are likely to need updating for new kernel versions, are also discouraged. Furthermore, a normal check for glibc headers updates needed for a new kernel version would only involve examining uapi headers (and the non-uapi linux/socket.h for new address families, an unfortunate existing wart in this area). As far as I can see, this value isn't defined in any uapi header, which makes it especially likely to be missed in such a check. Furthermore, I'm hoping to add more glibc tests for consistency of such constants between glibc and the kernel, to ensure any such updates missing are caught automatically through test failures - but that doesn't work if the constants in question aren't in a uapi header.
If this constant were in a uapi header, the glibc header could just include that - is the issue that it's not actually an interface between glibc and the kernel at all, but some kind of purely-userspace interface?
We very definitely wish to keep to a minimum the cases where updates need to be done separately in glibc by each architecture maintainer (that's just a recipe for some updates getting missed accidentally) - meaning that there needs to be a clear way in which someone can tell, globally for all architectures, whether the set of such architecture-specific headers for this constant in glibc is complete and current, and when it needs updating (and this should be as similar to possible to such checks for any other header constant).
Signature · 2 lines
Joseph S. Myers [email protected]
Quoted 9 lines · ----- On Jan 29, 2019, at 9:40 PM, Joseph Myers [email protected] wrote:
Quoted 4 lines · Mathieu Desnoyers
My thinking was to put the #error in the generic header, so architectures that are not supported yet cannot build against
rseq.hat all, so we don't end up in a broken upgrade scenario. I'm open to alternative ways to do it though, as long as we don't let not-yet-supported architectures build broken code.Any case with #error in installed glibc headers needs special-casing in
check-installed-headers.sh(and, thus, such errors are to be discouraged).
One alternative to #error would be to have an empty generic bits/rseq.h that does _not_ define RSEQ_SIG. This way, it would be possible to include sys/rseq.h from an architecture that does not define RSEQ_SIG yet, but it would not cause any build failure. It's only if the code try to use RSEQ_SIG that it would fail to compile because undefined.
Quoted 3 lines
Cases where architectures commonly need their own bits/ headers, especially where those are likely to need updating for new kernel versions, are also discouraged.
The per-arch bits/rseq.h headers, once they define a specific value for RSEQ_SIG, should never ever change that value.
Quoted 11 lines
Furthermore, a normal check for glibc
headers updates needed for a new kernel version would only involve examining uapi headers (and the non-uapilinux/socket.hfor new address families, an unfortunate existing wart in this area). As far as I can see, this value isn't defined in any uapi header, which makes it especially likely to be missed in such a check. Furthermore, I'm hoping to add more glibc tests for consistency of such constants between glibc and the kernel, to ensure any such updates missing are caught automatically through test failures - but that doesn't work if the constants in question aren't in a uapi header.
Quoted 3 lines
If this constant were in a uapi header, the glibc header could just include that - is the issue that it's not actually an interface between glibc and the kernel at all, but some kind of purely-userspace interface?
The rseq uapi headers do not enforce the value of RSEQ_SIG. The role of the kernel wrt signature is to receive it as sys_rseq argument, and then validate that abort targets are prefixed with the signature before moving the instruction pointer there.
Therefore, it's up to user-space to agree on the RSEQ_SIG value across all code using rseq within a process. Since glibc will be registering rseq and exposing public headers, it appears that glibc would be the appropriate project to define the RSEQ_SIG value for each architecture.
Quoted 9 lines
We very definitely wish to keep to a minimum the cases where updates need to be done separately in glibc by each architecture maintainer (that's just a recipe for some updates getting missed accidentally) - meaning that there needs to be a clear way in which someone can tell, globally for all architectures, whether the set of such architecture-specific headers for this constant in glibc is complete and current, and when it needs updating (and this should be as similar to possible to such checks for any other header constant).
Currently, I use #ifdef __NR_rseq from uapi unistd.h to check whether the kernel headers implement the rseq system call for the target architecture.
With the approach of having an empty bits/rseq.h for architecture not yet supporting rseq in glibc, one way to check that glibc implements RSEQ_SIG for all architectures that have the rseq system call wired up in uapi would be:
#include <unistd.h>
#include <sys/rseq.h>
#if defined (__NR_rseq) && !defined (RSEQ_SIG)
# error "UAPI headers support rseq system call, but glibc does not define RSEQ_SIG." #endif
Would that take care of your concerns ?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Quoted 5 lines · Mathieu Desnoyers
#if defined (
__NR_rseq) && !defined (RSEQ_SIG)# error "UAPI headers support rseq system call, but glibc does not define RSEQ_SIG." #endifWould that take care of your concerns ?
That would of course need appropriate conditionals based on the most recent kernel version for which a given glibc version has been updated, so that using new kernel headers with an existing glibc release does not make the build fail (cf. the test of syscall-names.list). And being able to write such a test only solves one half of the problem - it needs to be easy to determine what value to put in that header in glibc for an architecture that's newly gained support in the kernel, *without* needing any architecture expertise.
Signature · 2 lines
Joseph S. Myers [email protected]
Quoted 7 lines · ----- On Jan 30, 2019, at 4:10 PM, Joseph Myers [email protected] wrote:
Quoted 5 lines · Mathieu Desnoyers
#if defined (
__NR_rseq) && !defined (RSEQ_SIG)# error "UAPI headers support rseq system call, but glibc does not define RSEQ_SIG." #endif
Quoted 6 lines
Quoted 1 line
Would that take care of your concerns ?
That would of course need appropriate conditionals based on the most recent kernel version for which a given glibc version has been updated, so that using new kernel headers with an existing glibc release does not make the build fail (cf. the test of syscall-names.list).
The test I hint at above would not be for the glibc build per se. It would be for a check that glibc implements support for all the system calls available in the kernel headers (if such a test target currently exists).
Quoted 5 lines
And being able to
write such a test only solves one half of the problem - it needs to be easy to determine what value to put in that header in glibc for an architecture that's newly gained support in the kernel, *without* needing any architecture expertise.
I'm afraid this requirement is incompatible with the nature of the RSEQ signature. This signature may be required to be a specific trap instruction by the architecture, so deciding on its value without architecture expertise is not possible.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Quoted 26 lines · ----- On Jan 31, 2019, at 11:37 AM, Mathieu Desnoyers…
Quoted 7 lines · ----- On Jan 30, 2019, at 4:10 PM, Joseph Myers [email protected] wrote:
Quoted 5 lines · Mathieu Desnoyers
#if defined (
__NR_rseq) && !defined (RSEQ_SIG)# error "UAPI headers support rseq system call, but glibc does not define RSEQ_SIG." #endifQuoted 6 lines
Quoted 1 line
Would that take care of your concerns ?
That would of course need appropriate conditionals based on the most recent kernel version for which a given glibc version has been updated, so that using new kernel headers with an existing glibc release does not make the build fail (cf. the test of syscall-names.list).
The test I hint at above would not be for the glibc build per se. It would be for a check that glibc implements support for all the system calls available in the kernel headers (if such a test target currently exists).
Quoted 5 lines
And being able to
write such a test only solves one half of the problem - it needs to be easy to determine what value to put in that header in glibc for an architecture that's newly gained support in the kernel, *without* needing any architecture expertise.
Quoted 4 lines
I'm afraid this requirement is incompatible with the nature of the RSEQ signature. This signature may be required to be a specific trap instruction by the architecture, so deciding on its value without architecture expertise is not possible.
Just to clarify a point: the "success criterion" I'm aiming for here is to provide a rseq integration that does not cause foreseeable user crashes on upgrade.
I'm all for taking into account the maintenance burden on glibc maintainers as a metric in the implementation choices made, but at this point, I don't see how we can achieve success without introducing architecture headers for the RSEQ_SIG signature. If you have ideas on how to further minimize the maintenance burden for glibc maintainers while still meeting the success criterion, I'm all ears.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com