x86/entry/vdso: clean up the vdso build, vdso updates
Subsystems (10)
H. Peter Anvin
maintainerv4 cover x86/entry/vdso: clean up the vdso build, vdso updates
This patchset cleans up the vdso build by building the 32- and 64-bit vdsos in separate directories, moving the vdso2c tool to arch/x86/tools, and by merging common code and especially Makefile rules between the 32- and 64-bit vdsos to the greatest extent possible.
Be more strict about sanitizing and standardizing the vdso build options.
Disable CFI for the vdso until the kernel adds user space IBT support.
Modernize the DWARF generation vdso32/sigreturn.S.
Add macros to cleanly call system calls from vdso source code.
Add the GNU_PROPERTY and GNU_STACK PHDRs to the vdso.
When using int $0x80 (old 32-bit hardware or FRED-capable 64-bit hardware) skip the stack stuff in the 32-bit kernel entry code and call int $0x80 directly when used from C code.
Changes from v3 to v4:
- Improved description of patch 01/10.
- Split out the move of vdso2c to tools into a separate patch.
- Remove unrelated formatting changes from vdso-layout.lds.S.
- Fix *-x32.o being missing from "targets".
- Rebased onto v6.19-rc1.
Changes from v2 to v3:
In arch/x86/include/asm/vdso/sys_call.h:
- remove stray comment from prototyping (Uros Biszjak)
- remove VDSO_SYSCALL6() since it would require special handling on 32 bits with frame pointers and is currently unused. (Uros Biszjak)
- indent nested preprocessor directives.
Changes from v1 to v2:
Too many to count - much of the patchset has been reworked
Patches:
x86/entry/vdso: rename vdso_image_* to vdso*_image x86/entry/vdso: move vdso2c to arch/x86/tools
x86/entry/vdso: refactor the vdso build
x86/entry/vdso32: don't rely on int80_landing_pad for adjusting ip x86/entry/vdso32: remove SYSCALL_ENTER_KERNEL macro in sigreturn.S x86/entry/vdso32: remove open-coded DWARF in sigreturn.S x86/entry/vdso: include GNU_PROPERTY and GNU_STACK PHDRs x86/vdso: abstract out vdso system call internals x86/cpufeature: replace X86_FEATURE_SYSENTER32 with X86_FEATURE_SYSFAST32 x86/entry/vdso32: when using int $0x80, use it directly
---
arch/x86/Kconfig.cpufeatures | 8 +
arch/x86/Makefile | 2 +-
arch/x86/entry/syscall_32.c | 2 +-
arch/x86/entry/vdso/.gitignore | 11 +-
arch/x86/entry/vdso/Makefile | 162 +--------------------
arch/x86/entry/vdso/common/Makefile.include | 89 +++++++++++
arch/x86/entry/vdso/{vdso-note.S => common/note.S} | 5 +-
arch/x86/entry/vdso/{ => common}/vclock_gettime.c | 0
arch/x86/entry/vdso/{ => common}/vdso-layout.lds.S | 38 +++--
arch/x86/entry/vdso/{ => common}/vgetcpu.c | 0
arch/x86/entry/vdso/vdso32/Makefile | 24 +++
arch/x86/entry/vdso/vdso32/note.S | 19 +--
arch/x86/entry/vdso/vdso32/sigreturn.S | 152 +++++--------------
arch/x86/entry/vdso/vdso32/system_call.S | 22 ++-
arch/x86/entry/vdso/vdso32/vclock_gettime.c | 5 +-
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 2 +-
arch/x86/entry/vdso/vdso32/vgetcpu.c | 4 +-
arch/x86/entry/vdso/vdso64/Makefile | 46 ++++++
arch/x86/entry/vdso/vdso64/note.S | 1 +
arch/x86/entry/vdso/vdso64/vclock_gettime.c | 1 +
.../entry/vdso/{vdso.lds.S => vdso64/vdso64.lds.S} | 2 +-
arch/x86/entry/vdso/{ => vdso64}/vdsox32.lds.S | 2 +-
arch/x86/entry/vdso/vdso64/vgetcpu.c | 1 +
.../entry/vdso/{ => vdso64}/vgetrandom-chacha.S | 0
arch/x86/entry/vdso/{ => vdso64}/vgetrandom.c | 2 +-
arch/x86/entry/vdso/{ => vdso64}/vsgx.S | 0
arch/x86/entry/vdso/vma.c | 24 ++-
arch/x86/include/asm/cpufeatures.h | 2 +-
arch/x86/include/asm/dwarf2.h | 1 +
arch/x86/include/asm/elf.h | 2 +-
arch/x86/include/asm/vdso.h | 6 +-
arch/x86/include/asm/vdso/gettimeofday.h | 108 +-------------
arch/x86/include/asm/vdso/sys_call.h | 105 +++++++++++++
arch/x86/kernel/asm-offsets.c | 6 +
arch/x86/kernel/cpu/centaur.c | 3 -
arch/x86/kernel/cpu/common.c | 8 +
arch/x86/kernel/cpu/intel.c | 4 +-
arch/x86/kernel/cpu/zhaoxin.c | 4 +-
arch/x86/kernel/fred.c | 2 +-
arch/x86/kernel/process_64.c | 6 +-
arch/x86/kernel/signal_32.c | 4 +-
arch/x86/tools/Makefile | 15 +-
arch/x86/{entry/vdso => tools}/vdso2c.c | 0
arch/x86/{entry/vdso => tools}/vdso2c.h | 0
arch/x86/xen/setup.c | 28 ++--
arch/x86/xen/smp_pv.c | 5 +-
arch/x86/xen/xen-ops.h | 1 -
47 files changed, 444 insertions(+), 490 deletions(-)H. Peter Anvin
maintainerPATCH v4 1/10 x86/entry/vdso: rename vdso_image_* to vdso*_image
The vdso .so files are named vdso*.so. These structures are binary images and descriptions of these files, so it is more consistent for them to have a naming that more directly mirrors the filenames.
It is also very slightly more compact (by one character...) and simplifies the Makefile just a little bit.
--- arch/x86/entry/syscall_32.c | 2 +- arch/x86/entry/vdso/.gitignore | 11 ++++------- arch/x86/entry/vdso/Makefile | 8 ++++---- arch/x86/entry/vdso/vma.c | 10 +++++----- arch/x86/include/asm/elf.h | 2 +- arch/x86/include/asm/vdso.h | 6 +++--- arch/x86/kernel/process_64.c | 6 +++--- arch/x86/kernel/signal_32.c | 4 ++-- 8 files changed, 23 insertions(+), 26 deletions(-)
arch/x86/entry/syscall_32.c +1 -1
@@ -319,7 +319,7 @@ __visible noinstr bool do_fast_syscall_32(struct pt_regs *regs)* convention. Adjust regs so it looks like we entered using int80. */ unsigned long landing_pad = (unsigned long )current->mm->context.vdso + - vdso_image_32.sym_int80_landing_pad; + vdso32_image.sym_int80_landing_pad;/* * SYSENTER loses EIP, and even SYSCALL32 needs us to skip forward
arch/x86/entry/vdso/.gitignore +4 -7
@@ -1,8 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -vdso.lds -vdsox32.lds -vdso32-syscall-syms.lds -vdso32-sysenter-syms.lds -vdso32-int80-syms.lds -vdso-image-*.c -vdso2c +*.lds +*.so +*.so.dbg +vdso*-image.c
arch/x86/entry/vdso/Makefile +4 -4
@@ -16,9 +16,9 @@ vobjs-$(CONFIG_X86_SGX) += vsgx.o obj-y += vma.o extable.o # vDSO images to build: -obj-$(CONFIG_X86_64) += vdso-image-64.o -obj-$(CONFIG_X86_X32_ABI) += vdso-image-x32.o -obj-$(CONFIG_COMPAT_32) += vdso-image-32.o vdso32-setup.o +obj-$(CONFIG_X86_64) += vdso64-image.o +obj-$(CONFIG_X86_X32_ABI) += vdsox32-image.o +obj-$(CONFIG_COMPAT_32) += vdso32-image.o vdso32-setup.o vobjs := $(addprefix $(obj)/, $(vobjs-y)) vobjs32 := $(addprefix $(obj)/, $(vobjs32-y))
@@ -44,7 +44,7 @@ hostprogs += vdso2c quiet_cmd_vdso2c = VDSO2C $@ cmd_vdso2c = $(obj)/vdso2c $< $(<:%.dbg=%) $@ -$(obj)/vdso-image-%.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE +$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE $(call if_changed,vdso2c) #
arch/x86/entry/vdso/vma.c +5 -5
@@ -65,7 +65,7 @@ static vm_fault_t vdso_fault(const struct vm_special_mapping *sm,static void vdso_fix_landing (const struct vdso_image *image,struct vm_area_struct *new_vma) { -if (in_ia32_syscall () && image == &vdso_image_32) { +if (in_ia32_syscall () && image == &vdso32_image) {struct pt_regs *regs =current_pt_regs ();unsigned long vdso_land = image->sym_int80_landing_pad; unsigned long old_land_addr = vdso_land +
@@ -230,7 +230,7 @@ static int load_vdso32(void)if (vdso32_enabled !=1 )/* Other values all mean "disabled" */ return 0 ; -return map_vdso (&vdso_image_32,0 ); +return map_vdso (&vdso32_image,0 ); }int arch_setup_additional_pages (struct linux_binprm *bprm,int uses_interp)
@@ -239,7 +239,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)if (!vdso64_enabled)return 0 ; -return map_vdso (&vdso_image_64,0 ); +return map_vdso (&vdso64_image,0 ); }return load_vdso32 ();
@@ -252,7 +252,7 @@ int compat_arch_setup_additional_pages(struct linux_binprm *bprm,if (IS_ENABLED (CONFIG_X86_X32_ABI ) && x32) {if (!vdso64_enabled)return 0 ; -return map_vdso (&vdso_image_x32,0 ); +return map_vdso (&vdsox32_image,0 ); }if (IS_ENABLED (CONFIG_IA32_EMULATION ))
@@ -267,7 +267,7 @@ bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs)const struct vdso_image *image = current->mm->context.vdso_image;unsigned long vdso = (unsigned long ) current->mm->context.vdso; -if (in_ia32_syscall () && image == &vdso_image_32) { +if (in_ia32_syscall () && image == &vdso32_image) {if (regs->ip == vdso + image->sym_vdso32_sigreturn_landing_pad || regs->ip == vdso + image->sym_vdso32_rt_sigreturn_landing_pad)return true;
arch/x86/include/asm/elf.h +1 -1
@@ -361,7 +361,7 @@ else if (IS_ENABLED(CONFIG_IA32_EMULATION)) \#define VDSO_ENTRY \ ((unsigned long)current->mm->context.vdso + \ - vdso_image_32.sym___kernel_vsyscall) + vdso32_image.sym___kernel_vsyscall)struct linux_binprm ;
arch/x86/include/asm/vdso.h +3 -3
@@ -27,9 +27,9 @@ struct vdso_image {long sym_vdso32_rt_sigreturn_landing_pad; }; -extern const struct vdso_image vdso_image_64; -extern const struct vdso_image vdso_image_x32; -extern const struct vdso_image vdso_image_32; +extern const struct vdso_image vdso64_image; +extern const struct vdso_image vdsox32_image; +extern const struct vdso_image vdso32_image;extern int __init init_vdso_image (const struct vdso_image *image);
arch/x86/kernel/process_64.c +3 -3
@@ -941,14 +941,14 @@ long do_arch_prctl_64(struct task_struct *task, int option, unsigned long arg2)#ifdef CONFIG_CHECKPOINT_RESTORE # ifdef CONFIG_X86_X32_ABI case ARCH_MAP_VDSO_X32 : -return prctl_map_vdso (&vdso_image_x32, arg2); +return prctl_map_vdso (&vdsox32_image, arg2);# endif # ifdef CONFIG_IA32_EMULATION case ARCH_MAP_VDSO_32 : -return prctl_map_vdso (&vdso_image_32, arg2); +return prctl_map_vdso (&vdso32_image, arg2);# endif case ARCH_MAP_VDSO_64 : -return prctl_map_vdso (&vdso_image_64, arg2); +return prctl_map_vdso (&vdso64_image, arg2);#endif #ifdef CONFIG_ADDRESS_MASKING case ARCH_GET_UNTAG_MASK :
arch/x86/kernel/signal_32.c +2 -2
@@ -282,7 +282,7 @@ int ia32_setup_frame(struct ksignal *ksig, struct pt_regs *regs)/* Return stub is in 32bit vsyscall page */ if (current->mm->context.vdso) restorer = current->mm->context.vdso + - vdso_image_32.sym___kernel_sigreturn; + vdso32_image.sym___kernel_sigreturn;else restorer = &frame->retcode; }
@@ -368,7 +368,7 @@ int ia32_setup_rt_frame(struct ksignal *ksig, struct pt_regs *regs) restorer = ksig->ka.sa.sa_restorer; else restorer = current->mm->context.vdso + - vdso_image_32.sym___kernel_rt_sigreturn; + vdso32_image.sym___kernel_rt_sigreturn;unsafe_put_user (ptr_to_compat (restorer), &frame->pretcode, Efault);/*
Signature · 1 line
2.52.0
[tip: x86/entry] x86/entry/vdso: Rename vdso_image_* to vdso*_image
The following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: 93d73005bff4f600696ce30e366e742c3373b13d Gitweb: https://git.kernel.org/tip/93d73005bff4f600696ce30e366e742c3373b13d Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:25:55 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 15:33:20 -08:00
x86/entry/vdso: Rename vdso_image_* to vdso*_image
The vdso .so files are named vdso*.so. These structures are binary images and descriptions of these files, so it is more consistent for them to have a naming that more directly mirrors the filenames.
It is also very slightly more compact (by one character...) and simplifies the Makefile just a little bit.
--- arch/x86/entry/syscall_32.c | 2 +- arch/x86/entry/vdso/.gitignore | 11 ++++------- arch/x86/entry/vdso/Makefile | 8 ++++---- arch/x86/entry/vdso/vma.c | 10 +++++----- arch/x86/include/asm/elf.h | 2 +- arch/x86/include/asm/vdso.h | 6 +++--- arch/x86/kernel/process_64.c | 6 +++--- arch/x86/kernel/signal_32.c | 4 ++-- 8 files changed, 23 insertions(+), 26 deletions(-)
arch/x86/entry/syscall_32.c +1 -1
@@ -319,7 +319,7 @@ __visible noinstr bool do_fast_syscall_32(struct pt_regs *regs)* convention. Adjust regs so it looks like we entered using int80. */ unsigned long landing_pad = (unsigned long )current->mm->context.vdso + - vdso_image_32.sym_int80_landing_pad; + vdso32_image.sym_int80_landing_pad;/* * SYSENTER loses EIP, and even SYSCALL32 needs us to skip forward
arch/x86/entry/vdso/.gitignore +4 -7
@@ -1,8 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -vdso.lds -vdsox32.lds -vdso32-syscall-syms.lds -vdso32-sysenter-syms.lds -vdso32-int80-syms.lds -vdso-image-*.c -vdso2c +*.lds +*.so +*.so.dbg +vdso*-image.c
arch/x86/entry/vdso/Makefile +4 -4
@@ -16,9 +16,9 @@ vobjs-$(CONFIG_X86_SGX) += vsgx.o obj-y += vma.o extable.o # vDSO images to build: -obj-$(CONFIG_X86_64) += vdso-image-64.o -obj-$(CONFIG_X86_X32_ABI) += vdso-image-x32.o -obj-$(CONFIG_COMPAT_32) += vdso-image-32.o vdso32-setup.o +obj-$(CONFIG_X86_64) += vdso64-image.o +obj-$(CONFIG_X86_X32_ABI) += vdsox32-image.o +obj-$(CONFIG_COMPAT_32) += vdso32-image.o vdso32-setup.o vobjs := $(addprefix $(obj)/, $(vobjs-y)) vobjs32 := $(addprefix $(obj)/, $(vobjs32-y))
@@ -44,7 +44,7 @@ hostprogs += vdso2c quiet_cmd_vdso2c = VDSO2C $@ cmd_vdso2c = $(obj)/vdso2c $< $(<:%.dbg=%) $@ -$(obj)/vdso-image-%.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE +$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE $(call if_changed,vdso2c) #
arch/x86/entry/vdso/vma.c +5 -5
@@ -65,7 +65,7 @@ static vm_fault_t vdso_fault(const struct vm_special_mapping *sm,static void vdso_fix_landing (const struct vdso_image *image,struct vm_area_struct *new_vma) { -if (in_ia32_syscall () && image == &vdso_image_32) { +if (in_ia32_syscall () && image == &vdso32_image) {struct pt_regs *regs =current_pt_regs ();unsigned long vdso_land = image->sym_int80_landing_pad; unsigned long old_land_addr = vdso_land +
@@ -230,7 +230,7 @@ static int load_vdso32(void)if (vdso32_enabled !=1 )/* Other values all mean "disabled" */ return 0 ; -return map_vdso (&vdso_image_32,0 ); +return map_vdso (&vdso32_image,0 ); }int arch_setup_additional_pages (struct linux_binprm *bprm,int uses_interp)
@@ -239,7 +239,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)if (!vdso64_enabled)return 0 ; -return map_vdso (&vdso_image_64,0 ); +return map_vdso (&vdso64_image,0 ); }return load_vdso32 ();
@@ -252,7 +252,7 @@ int compat_arch_setup_additional_pages(struct linux_binprm *bprm,if (IS_ENABLED (CONFIG_X86_X32_ABI ) && x32) {if (!vdso64_enabled)return 0 ; -return map_vdso (&vdso_image_x32,0 ); +return map_vdso (&vdsox32_image,0 ); }if (IS_ENABLED (CONFIG_IA32_EMULATION ))
@@ -267,7 +267,7 @@ bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs)const struct vdso_image *image = current->mm->context.vdso_image;unsigned long vdso = (unsigned long ) current->mm->context.vdso; -if (in_ia32_syscall () && image == &vdso_image_32) { +if (in_ia32_syscall () && image == &vdso32_image) {if (regs->ip == vdso + image->sym_vdso32_sigreturn_landing_pad || regs->ip == vdso + image->sym_vdso32_rt_sigreturn_landing_pad)return true;
arch/x86/include/asm/elf.h +1 -1
@@ -361,7 +361,7 @@ else if (IS_ENABLED(CONFIG_IA32_EMULATION)) \#define VDSO_ENTRY \ ((unsigned long)current->mm->context.vdso + \ - vdso_image_32.sym___kernel_vsyscall) + vdso32_image.sym___kernel_vsyscall)struct linux_binprm ;
arch/x86/include/asm/vdso.h +3 -3
@@ -27,9 +27,9 @@ struct vdso_image {long sym_vdso32_rt_sigreturn_landing_pad; }; -extern const struct vdso_image vdso_image_64; -extern const struct vdso_image vdso_image_x32; -extern const struct vdso_image vdso_image_32; +extern const struct vdso_image vdso64_image; +extern const struct vdso_image vdsox32_image; +extern const struct vdso_image vdso32_image;extern int __init init_vdso_image (const struct vdso_image *image);
arch/x86/kernel/process_64.c +3 -3
@@ -941,14 +941,14 @@ long do_arch_prctl_64(struct task_struct *task, int option, unsigned long arg2)#ifdef CONFIG_CHECKPOINT_RESTORE # ifdef CONFIG_X86_X32_ABI case ARCH_MAP_VDSO_X32 : -return prctl_map_vdso (&vdso_image_x32, arg2); +return prctl_map_vdso (&vdsox32_image, arg2);# endif # ifdef CONFIG_IA32_EMULATION case ARCH_MAP_VDSO_32 : -return prctl_map_vdso (&vdso_image_32, arg2); +return prctl_map_vdso (&vdso32_image, arg2);# endif case ARCH_MAP_VDSO_64 : -return prctl_map_vdso (&vdso_image_64, arg2); +return prctl_map_vdso (&vdso64_image, arg2);#endif #ifdef CONFIG_ADDRESS_MASKING case ARCH_GET_UNTAG_MASK :
arch/x86/kernel/signal_32.c +2 -2
@@ -282,7 +282,7 @@ int ia32_setup_frame(struct ksignal *ksig, struct pt_regs *regs)/* Return stub is in 32bit vsyscall page */ if (current->mm->context.vdso) restorer = current->mm->context.vdso + - vdso_image_32.sym___kernel_sigreturn; + vdso32_image.sym___kernel_sigreturn;else restorer = &frame->retcode; }
@@ -368,7 +368,7 @@ int ia32_setup_rt_frame(struct ksignal *ksig, struct pt_regs *regs) restorer = ksig->ka.sa.sa_restorer; else restorer = current->mm->context.vdso + - vdso_image_32.sym___kernel_rt_sigreturn; + vdso32_image.sym___kernel_rt_sigreturn;unsafe_put_user (ptr_to_compat (restorer), &frame->pretcode, Efault);/*
H. Peter Anvin
maintainerPATCH v4 2/10 x86/entry/vdso: move vdso2c to arch/x86/tools
In reply to H. Peter AnvinIt is generally better to build tools in arch/x86/tools to keep host cflags proliferation down, and to reduce makefile sequencing issues. Move the vdso build tool vdso2c into arch/x86/tools in preparation for refactoring the vdso makefiles.
---
arch/x86/Makefile | 2 +-
arch/x86/entry/vdso/Makefile | 7 +++----
arch/x86/tools/Makefile | 15 ++++++++++-----
arch/x86/{entry/vdso => tools}/vdso2c.c | 0
arch/x86/{entry/vdso => tools}/vdso2c.h | 0
5 files changed, 14 insertions(+), 10 deletions(-)
rename arch/x86/{entry/vdso => tools}/vdso2c.c (100%)
rename arch/x86/{entry/vdso => tools}/vdso2c.h (100%)arch/x86/Makefile +1 -1
@@ -252,7 +252,7 @@ endif archscripts: scripts_basic - $(Q)$(MAKE) $(build)=arch/x86/tools relocs + $(Q)$(MAKE) $(build)=arch/x86/tools relocs vdso2c ### # Syscall table generation
arch/x86/entry/vdso/Makefile +3 -4
@@ -38,13 +38,12 @@ VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 \ $(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE $(call if_changed,vdso_and_check) -HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi -I$(srctree)/arch/$(SUBARCH)/include/uapi -hostprogs += vdso2c +VDSO2C = $(objtree)/arch/x86/tools/vdso2c quiet_cmd_vdso2c = VDSO2C $@ - cmd_vdso2c = $(obj)/vdso2c $< $(<:%.dbg=%) $@ + cmd_vdso2c = $(VDSO2C) $< $(<:%.dbg=%) $@ -$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE +$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(VDSO2C) FORCE $(call if_changed,vdso2c) #
arch/x86/tools/Makefile +10 -5
@@ -38,9 +38,14 @@ $(obj)/insn_decoder_test.o: $(srctree)/tools/arch/x86/lib/insn.c $(srctree)/tool $(obj)/insn_sanity.o: $(srctree)/tools/arch/x86/lib/insn.c $(srctree)/tools/arch/x86/lib/inat.c $(srctree)/tools/arch/x86/include/asm/inat_types.h $(srctree)/tools/arch/x86/include/asm/inat.h $(srctree)/tools/arch/x86/include/asm/insn.h $(objtree)/arch/x86/lib/inat-tables.c -HOST_EXTRACFLAGS += -I$(srctree)/tools/include -hostprogs += relocs -relocs-objs := relocs_32.o relocs_64.o relocs_common.o -PHONY += relocs -relocs: $(obj)/relocs +HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi \ + -I$(srctree)/arch/$(SUBARCH)/include/uapi + +hostprogs += relocs vdso2c +relocs-objs := relocs_32.o relocs_64.o relocs_common.o + +always-y := $(hostprogs) + +PHONY += $(hostprogs) +$(hostprogs): %: $(obj)/% @:
arch/x86/tools/vdso2c.c renamed from arch/x86/entry/vdso/vdso2c.c +0 -0
arch/x86/tools/vdso2c.h renamed from arch/x86/entry/vdso/vdso2c.h +0 -0
Signature · 1 line
2.52.0
[tip: x86/entry] x86/entry/vdso: Move vdso2c to arch/x86/tools
The following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: a76108d05ee13cddb72b620752a80b2c3e87aee1 Gitweb: https://git.kernel.org/tip/a76108d05ee13cddb72b620752a80b2c3e87aee1 Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:25:56 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 15:33:20 -08:00
x86/entry/vdso: Move vdso2c to arch/x86/tools
It is generally better to build tools in arch/x86/tools to keep host cflags proliferation down, and to reduce makefile sequencing issues. Move the vdso build tool vdso2c into arch/x86/tools in preparation for refactoring the vdso makefiles.
--- arch/x86/Makefile | 2 +- arch/x86/entry/vdso/Makefile | 7 +- arch/x86/entry/vdso/vdso2c.c | 233 +---------------------------------- arch/x86/entry/vdso/vdso2c.h | 208 +------------------------------ arch/x86/tools/Makefile | 15 +- arch/x86/tools/vdso2c.c | 233 ++++++++++++++++++++++++++++++++++- arch/x86/tools/vdso2c.h | 208 ++++++++++++++++++++++++++++++- 7 files changed, 455 insertions(+), 451 deletions(-) delete mode 100644 arch/x86/entry/vdso/vdso2c.c delete mode 100644 arch/x86/entry/vdso/vdso2c.h create mode 100644 arch/x86/tools/vdso2c.c create mode 100644 arch/x86/tools/vdso2c.h
arch/x86/Makefile +1 -1
@@ -252,7 +252,7 @@ endif archscripts: scripts_basic - $(Q)$(MAKE) $(build)=arch/x86/tools relocs + $(Q)$(MAKE) $(build)=arch/x86/tools relocs vdso2c ### # Syscall table generation
arch/x86/entry/vdso/Makefile +3 -4
@@ -38,13 +38,12 @@ VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 \ $(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE $(call if_changed,vdso_and_check) -HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi -I$(srctree)/arch/$(SUBARCH)/include/uapi -hostprogs += vdso2c +VDSO2C = $(objtree)/arch/x86/tools/vdso2c quiet_cmd_vdso2c = VDSO2C $@ - cmd_vdso2c = $(obj)/vdso2c $< $(<:%.dbg=%) $@ + cmd_vdso2c = $(VDSO2C) $< $(<:%.dbg=%) $@ -$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE +$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(VDSO2C) FORCE $(call if_changed,vdso2c) #
arch/x86/entry/vdso/vdso2c.c deleted +0 -233
233 lines not on this page · raw message
arch/x86/entry/vdso/vdso2c.h deleted +0 -208
208 lines not on this page · raw message
arch/x86/tools/Makefile +10 -5
@@ -38,9 +38,14 @@ $(obj)/insn_decoder_test.o: $(srctree)/tools/arch/x86/lib/insn.c $(srctree)/tool $(obj)/insn_sanity.o: $(srctree)/tools/arch/x86/lib/insn.c $(srctree)/tools/arch/x86/lib/inat.c $(srctree)/tools/arch/x86/include/asm/inat_types.h $(srctree)/tools/arch/x86/include/asm/inat.h $(srctree)/tools/arch/x86/include/asm/insn.h $(objtree)/arch/x86/lib/inat-tables.c -HOST_EXTRACFLAGS += -I$(srctree)/tools/include -hostprogs += relocs -relocs-objs := relocs_32.o relocs_64.o relocs_common.o -PHONY += relocs -relocs: $(obj)/relocs +HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi \ + -I$(srctree)/arch/$(SUBARCH)/include/uapi + +hostprogs += relocs vdso2c +relocs-objs := relocs_32.o relocs_64.o relocs_common.o + +always-y := $(hostprogs) + +PHONY += $(hostprogs) +$(hostprogs): %: $(obj)/% @:
arch/x86/tools/vdso2c.c new file +233 -0
233 lines not on this page · raw message
arch/x86/tools/vdso2c.h new file +208 -0
208 lines not on this page · raw message
Borislav Petkov
maintainerPATCH 1/1 x86/entry/vdso: Add vdso2c to .gitignore
From: "Borislav Petkov (AMD)" <[email protected]>
Date: Tue, 27 Jan 2026 23:09:13 +0100
The commit
a76108d05ee1 ("x86/entry/vdso: Move vdso2c to arch/x86/tools")
moved vdso2c to arch/x86/tools/ and commit
93d73005bff4 ("x86/entry/vdso: Rename vdso_image_* to vdso*_image")
renamed .so files but also dropped vdso2c fromarch/x86/entry/vdso/.gitignore.
It should've moved it to arch/x86/tools/.gitignore instead.
Do that.
--- arch/x86/tools/.gitignore | 1 + 1 file changed, 1 insertion(+)
arch/x86/tools/.gitignore +1 -0
@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only relocs +vdso2c
Signature · 8 lines
2.51.0
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette[tip: x86/entry] x86/entry/vdso: Add vdso2c to .gitignore
The following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: ce9b1c10c3f1c723c3cc7b63aa8331fdb6c57a04 Gitweb: https://git.kernel.org/tip/ce9b1c10c3f1c723c3cc7b63aa8331fdb6c57a04 Author: Borislav Petkov (AMD) <[email protected]> AuthorDate: Tue, 27 Jan 2026 23:09:13 +01:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Tue, 27 Jan 2026 23:27:51 +01:00
x86/entry/vdso: Add vdso2c to .gitignore
The commit
a76108d05ee1 ("x86/entry/vdso: Move vdso2c to arch/x86/tools")
moved vdso2c to arch/x86/tools/ and commit
93d73005bff4 ("x86/entry/vdso: Rename vdso_image_* to vdso*_image")
renamed .so files but also dropped vdso2c fromarch/x86/entry/vdso/.gitignore.
It should've moved it to arch/x86/tools/.gitignore instead.
Do that.
--- arch/x86/tools/.gitignore | 1 + 1 file changed, 1 insertion(+)
arch/x86/tools/.gitignore +1 -0
@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only relocs +vdso2c
H. Peter Anvin
maintainerPATCH v4 3/10 x86/entry/vdso: refactor the vdso build
In reply to H. Peter Anvin- Separate out the vdso sources into common, vdso32, and vdso64 directories.
- Build the 32- and 64-bit vdsos in their respective subdirectories; this greatly simplifies the build flags handling.
- Unify the mangling of Makefile flags between the 32- and 64-bit vdso code as much as possible; all common rules are put in arch/x86/entry/vdso/common/Makefile.include. The remaining is very simple for 32 bits; the 64-bit one is only slightly more complicated because it contains the x32 generation rule.
- Define __DISABLE_EXPORTS when building the vdso. This need seems to have been masked by different ordering compile flags before.
- Change CONFIG_X86_64 to BUILD_VDSO32_64 in vdso32/system_call.S, to make it compatible with including fake_32bit_build.h.
- The -fcf-protection= option was "leaking" from the kernel build, for reasons that was not clear to me. Futhermore, several distributions ship with it set to a default value other than "-fcf-protection=none". Make it match the configuration options for *user space*.
Note that this patch may seem large, but the vast majority of it is simply code movement.
---
arch/x86/entry/vdso/Makefile | 161 +-----------------
arch/x86/entry/vdso/common/Makefile.include | 89 ++++++++++
.../entry/vdso/{vdso-note.S => common/note.S} | 5 +-
.../entry/vdso/{ => common}/vclock_gettime.c | 0
.../entry/vdso/{ => common}/vdso-layout.lds.S | 0
arch/x86/entry/vdso/{ => common}/vgetcpu.c | 0
arch/x86/entry/vdso/vdso32/Makefile | 24 +++
arch/x86/entry/vdso/vdso32/note.S | 19 +--
arch/x86/entry/vdso/vdso32/system_call.S | 2 +-
arch/x86/entry/vdso/vdso32/vclock_gettime.c | 5 +-
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 2 +-
arch/x86/entry/vdso/vdso32/vgetcpu.c | 4 +-
arch/x86/entry/vdso/vdso64/Makefile | 46 +++++
arch/x86/entry/vdso/vdso64/note.S | 1 +
arch/x86/entry/vdso/vdso64/vclock_gettime.c | 1 +
.../vdso/{vdso.lds.S => vdso64/vdso64.lds.S} | 2 +-
.../x86/entry/vdso/{ => vdso64}/vdsox32.lds.S | 2 +-
arch/x86/entry/vdso/vdso64/vgetcpu.c | 1 +
.../vdso/{ => vdso64}/vgetrandom-chacha.S | 0
arch/x86/entry/vdso/{ => vdso64}/vgetrandom.c | 2 +-
arch/x86/entry/vdso/{ => vdso64}/vsgx.S | 0
21 files changed, 180 insertions(+), 186 deletions(-)
create mode 100644 arch/x86/entry/vdso/common/Makefile.include
rename arch/x86/entry/vdso/{vdso-note.S => common/note.S} (62%)
rename arch/x86/entry/vdso/{ => common}/vclock_gettime.c (100%)
rename arch/x86/entry/vdso/{ => common}/vdso-layout.lds.S (100%)
rename arch/x86/entry/vdso/{ => common}/vgetcpu.c (100%)
create mode 100644 arch/x86/entry/vdso/vdso32/Makefile
create mode 100644 arch/x86/entry/vdso/vdso64/Makefile
create mode 100644 arch/x86/entry/vdso/vdso64/note.S
create mode 100644 arch/x86/entry/vdso/vdso64/vclock_gettime.c
rename arch/x86/entry/vdso/{vdso.lds.S => vdso64/vdso64.lds.S} (94%)
rename arch/x86/entry/vdso/{ => vdso64}/vdsox32.lds.S (92%)
create mode 100644 arch/x86/entry/vdso/vdso64/vgetcpu.c
rename arch/x86/entry/vdso/{ => vdso64}/vgetrandom-chacha.S (100%)
rename arch/x86/entry/vdso/{ => vdso64}/vgetrandom.c (91%)
rename arch/x86/entry/vdso/{ => vdso64}/vsgx.S (100%)arch/x86/entry/vdso/Makefile +6 -155
@@ -3,159 +3,10 @@ # Building vDSO images for x86. # -# Include the generic Makefile to check the built vDSO: -include $(srctree)/lib/vdso/Makefile.include +# Regular kernel objects +obj-y := vma.o extable.o +obj-$(CONFIG_COMPAT_32) += vdso32-setup.o -# Files to link into the vDSO: -vobjs-y := vdso-note.o vclock_gettime.o vgetcpu.o vgetrandom.o vgetrandom-chacha.o -vobjs32-y := vdso32/note.o vdso32/system_call.o vdso32/sigreturn.o -vobjs32-y += vdso32/vclock_gettime.o vdso32/vgetcpu.o -vobjs-$(CONFIG_X86_SGX) += vsgx.o - -# Files to link into the kernel: -obj-y += vma.o extable.o - -# vDSO images to build: -obj-$(CONFIG_X86_64) += vdso64-image.o -obj-$(CONFIG_X86_X32_ABI) += vdsox32-image.o -obj-$(CONFIG_COMPAT_32) += vdso32-image.o vdso32-setup.o - -vobjs := $(addprefix $(obj)/, $(vobjs-y)) -vobjs32 := $(addprefix $(obj)/, $(vobjs32-y)) - -$(obj)/vdso.o: $(obj)/vdso.so - -targets += vdso.lds $(vobjs-y) -targets += vdso32/vdso32.lds $(vobjs32-y) - -targets += $(foreach x, 64 x32 32, vdso-image-$(x).c vdso$(x).so vdso$(x).so.dbg) - -CPPFLAGS_vdso.lds += -P -C - -VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 \ - -z max-page-size=4096 - -$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE - $(call if_changed,vdso_and_check) - -VDSO2C = $(objtree)/arch/x86/tools/vdso2c - -quiet_cmd_vdso2c = VDSO2C $@ - cmd_vdso2c = $(VDSO2C) $< $(<:%.dbg=%) $@ - -$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(VDSO2C) FORCE - $(call if_changed,vdso2c) - -# -# Don't omit frame pointers for ease of userspace debugging, but do -# optimize sibling calls. -# -CFL := $(PROFILING) -mcmodel=small -fPIC -O2 -fasynchronous-unwind-tables -m64 \ - $(filter -g%,$(KBUILD_CFLAGS)) -fno-stack-protector \ - -fno-omit-frame-pointer -foptimize-sibling-calls \ - -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO - -ifdef CONFIG_MITIGATION_RETPOLINE -ifneq ($(RETPOLINE_VDSO_CFLAGS),) - CFL += $(RETPOLINE_VDSO_CFLAGS) -endif -endif - -$(vobjs): KBUILD_CFLAGS := $(filter-out $(PADDING_CFLAGS) $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) $(RANDSTRUCT_CFLAGS) $(KSTACK_ERASE_CFLAGS) $(GCC_PLUGINS_CFLAGS) $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS)) $(CFL) -$(vobjs): KBUILD_AFLAGS += -DBUILD_VDSO - -# -# vDSO code runs in userspace and -pg doesn't help with profiling anyway. -# -CFLAGS_REMOVE_vclock_gettime.o = -pg -CFLAGS_REMOVE_vdso32/vclock_gettime.o = -pg -CFLAGS_REMOVE_vgetcpu.o = -pg -CFLAGS_REMOVE_vdso32/vgetcpu.o = -pg -CFLAGS_REMOVE_vsgx.o = -pg -CFLAGS_REMOVE_vgetrandom.o = -pg - -# -# X32 processes use x32 vDSO to access 64bit kernel data. -# -# Build x32 vDSO image: -# 1. Compile x32 vDSO as 64bit. -# 2. Convert object files to x32. -# 3. Build x32 VDSO image with x32 objects, which contains 64bit codes -# so that it can reach 64bit address space with 64bit pointers. -# - -CPPFLAGS_vdsox32.lds = $(CPPFLAGS_vdso.lds) -VDSO_LDFLAGS_vdsox32.lds = -m elf32_x86_64 -soname linux-vdso.so.1 \ - -z max-page-size=4096 - -# x32-rebranded versions -vobjx32s-y := $(vobjs-y:.o=-x32.o) - -# same thing, but in the output directory -vobjx32s := $(addprefix $(obj)/, $(vobjx32s-y)) - -# Convert 64bit object file to x32 for x32 vDSO. -quiet_cmd_x32 = X32 $@ - cmd_x32 = $(OBJCOPY) -O elf32-x86-64 $< $@ - -$(obj)/%-x32.o: $(obj)/%.o FORCE - $(call if_changed,x32) - -targets += vdsox32.lds $(vobjx32s-y) - -$(obj)/%.so: OBJCOPYFLAGS := -S --remove-section __ex_table -$(obj)/%.so: $(obj)/%.so.dbg FORCE - $(call if_changed,objcopy) - -$(obj)/vdsox32.so.dbg: $(obj)/vdsox32.lds $(vobjx32s) FORCE - $(call if_changed,vdso_and_check) - -CPPFLAGS_vdso32/vdso32.lds = $(CPPFLAGS_vdso.lds) -VDSO_LDFLAGS_vdso32.lds = -m elf_i386 -soname linux-gate.so.1 - -KBUILD_AFLAGS_32 := $(filter-out -m64,$(KBUILD_AFLAGS)) -DBUILD_VDSO -$(obj)/vdso32.so.dbg: KBUILD_AFLAGS = $(KBUILD_AFLAGS_32) -$(obj)/vdso32.so.dbg: asflags-$(CONFIG_X86_64) += -m32 - -KBUILD_CFLAGS_32 := $(filter-out -m64,$(KBUILD_CFLAGS)) -KBUILD_CFLAGS_32 := $(filter-out -mcmodel=kernel,$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out -fno-pic,$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out -mfentry,$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(RANDSTRUCT_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(KSTACK_ERASE_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(GCC_PLUGINS_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(CC_FLAGS_CFI),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(PADDING_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 += -m32 -msoft-float -mregparm=0 -fpic -KBUILD_CFLAGS_32 += -fno-stack-protector -KBUILD_CFLAGS_32 += $(call cc-option, -foptimize-sibling-calls) -KBUILD_CFLAGS_32 += -fno-omit-frame-pointer -KBUILD_CFLAGS_32 += -DDISABLE_BRANCH_PROFILING -KBUILD_CFLAGS_32 += -DBUILD_VDSO - -ifdef CONFIG_MITIGATION_RETPOLINE -ifneq ($(RETPOLINE_VDSO_CFLAGS),) - KBUILD_CFLAGS_32 += $(RETPOLINE_VDSO_CFLAGS) -endif -endif - -$(obj)/vdso32.so.dbg: KBUILD_CFLAGS = $(KBUILD_CFLAGS_32) - -$(obj)/vdso32.so.dbg: $(obj)/vdso32/vdso32.lds $(vobjs32) FORCE - $(call if_changed,vdso_and_check) - -# -# The DSO images are built using a special linker script. -# -quiet_cmd_vdso = VDSO $@ - cmd_vdso = $(LD) -o $@ \ - $(VDSO_LDFLAGS) $(VDSO_LDFLAGS_$(filter %.lds,$(^F))) \ - -T $(filter %.lds,$^) $(filter %.o,$^) - -VDSO_LDFLAGS = -shared --hash-style=both --build-id=sha1 --no-undefined \ - $(call ld-option, --eh-frame-hdr) -Bsymbolic -z noexecstack - -quiet_cmd_vdso_and_check = VDSO $@ - cmd_vdso_and_check = $(cmd_vdso); $(cmd_vdso_check) +# vDSO directories +obj-$(CONFIG_X86_64) += vdso64/ +obj-$(CONFIG_COMPAT_32) += vdso32/
arch/x86/entry/vdso/common/Makefile.include new file +89 -0
89 lines not on this page · raw message
arch/x86/entry/vdso/common/note.S renamed from arch/x86/entry/vdso/vdso-note.S +4 -1
@@ -1,13 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ /* * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text. * Here we can supply some information useful to userland. */ #include <linux/build-salt.h> -#include <linux/uts.h> #include <linux/version.h> #include <linux/elfnote.h> +/* Ideally this would use UTS_NAME, but using a quoted string here + doesn't work. Remember to change this when changing the + kernel's name. */ ELFNOTE_START(Linux, 0, "a") .long LINUX_VERSION_CODE ELFNOTE_END
arch/x86/entry/vdso/common/vclock_gettime.c renamed from arch/x86/entry/vdso/vclock_gettime.c +0 -0
arch/x86/entry/vdso/common/vdso-layout.lds.S renamed from arch/x86/entry/vdso/vdso-layout.lds.S +0 -0
arch/x86/entry/vdso/common/vgetcpu.c renamed from arch/x86/entry/vdso/vgetcpu.c +0 -0
arch/x86/entry/vdso/vdso32/Makefile new file +24 -0
@@ -0,0 +1,24 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# 32-bit vDSO images for x86. +# + +# The vDSOs built in this directory +vdsos-y := 32 + +# Files to link into the vDSO: +vobjs-y := note.o vclock_gettime.o vgetcpu.o +vobjs-y += system_call.o sigreturn.o + +# Compilation flags +flags-y := -DBUILD_VDSO32 -m32 -mregparm=0 +flags-$(CONFIG_X86_64) += -include $(src)/fake_32bit_build.h +flags-remove-y := -m64 + +# The location of this include matters! +include $(src)/../common/Makefile.include + +# Linker options for the vdso +VDSO_LDFLAGS_32 := -m elf_i386 -soname linux-gate.so.1 + +$(obj)/vdso32.so.dbg: $(vobjs)
arch/x86/entry/vdso/vdso32/note.S +1 -18
@@ -1,18 +1 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text. - * Here we can supply some information useful to userland. - */ - -#include <linux/build-salt.h> -#include <linux/version.h> -#include <linux/elfnote.h> - -/* Ideally this would use UTS_NAME, but using a quoted string here - doesn't work. Remember to change this when changing the - kernel's name. */ -ELFNOTE_START(Linux, 0, "a") - .long LINUX_VERSION_CODE -ELFNOTE_END - -BUILD_SALT +#include "common/note.S"
arch/x86/entry/vdso/vdso32/system_call.S +1 -1
@@ -52,7 +52,7 @@ __kernel_vsyscall: #define SYSENTER_SEQUENCE "movl %esp, %ebp; sysenter" #define SYSCALL_SEQUENCE "movl %ecx, %ebp; syscall" -#ifdef CONFIG_X86_64 +#ifdef BUILD_VDSO32_64 /* If SYSENTER (Intel) or SYSCALL32 (AMD) is available, use it. */ ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSENTER32, \ SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32
arch/x86/entry/vdso/vdso32/vclock_gettime.c +1 -4
@@ -1,4 +1 @@ -// SPDX-License-Identifier: GPL-2.0 -#define BUILD_VDSO32 -#include "fake_32bit_build.h" -#include "../vclock_gettime.c" +#include "common/vclock_gettime.c"
arch/x86/entry/vdso/vdso32/vdso32.lds.S +1 -1
@@ -11,7 +11,7 @@ #define BUILD_VDSO32 -#include "../vdso-layout.lds.S" +#include "common/vdso-layout.lds.S" /* The ELF entry point can be used to set the AT_SYSINFO value. */ ENTRY(__kernel_vsyscall);
arch/x86/entry/vdso/vdso32/vgetcpu.c +1 -3
@@ -1,3 +1 @@ -// SPDX-License-Identifier: GPL-2.0 -#include "fake_32bit_build.h" -#include "../vgetcpu.c" +#include "common/vgetcpu.c"
arch/x86/entry/vdso/vdso64/Makefile new file +46 -0
46 lines not on this page · raw message
arch/x86/entry/vdso/vdso64/note.S new file +1 -0
@@ -0,0 +1 @@ +#include "common/note.S"
arch/x86/entry/vdso/vdso64/vclock_gettime.c new file +1 -0
@@ -0,0 +1 @@ +#include "common/vclock_gettime.c"
arch/x86/entry/vdso/vdso64/vdso64.lds.S renamed from arch/x86/entry/vdso/vdso.lds.S +1 -1
@@ -9,7 +9,7 @@ #define BUILD_VDSO64 -#include "vdso-layout.lds.S" +#include "common/vdso-layout.lds.S" /* * This controls what userland symbols we export from the vDSO.
arch/x86/entry/vdso/vdso64/vdsox32.lds.S renamed from arch/x86/entry/vdso/vdsox32.lds.S +1 -1
@@ -9,7 +9,7 @@ #define BUILD_VDSOX32 -#include "vdso-layout.lds.S" +#include "common/vdso-layout.lds.S" /* * This controls what userland symbols we export from the vDSO.
arch/x86/entry/vdso/vdso64/vgetcpu.c new file +1 -0
@@ -0,0 +1 @@ +#include "common/vgetcpu.c"
arch/x86/entry/vdso/vdso64/vgetrandom-chacha.S renamed from arch/x86/entry/vdso/vgetrandom-chacha.S +0 -0
arch/x86/entry/vdso/vdso64/vgetrandom.c renamed from arch/x86/entry/vdso/vgetrandom.c +1 -1
@@ -4,7 +4,7 @@*/ #include <linux/types.h> -#include "../../../../lib/vdso/getrandom.c" +#include "lib/vdso/getrandom.c" ssize_t __vdso_getrandom (void *buffer,size_t len,unsigned int flags,void *opaque_state,size_t opaque_len) {
arch/x86/entry/vdso/vdso64/vsgx.S renamed from arch/x86/entry/vdso/vsgx.S +0 -0
Signature · 1 line
2.52.0
[tip: x86/entry] x86/entry/vdso: Refactor the vdso build
The following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: 693c819fedcdcabfda7488e2d5e355a84c2fd1b0 Gitweb: https://git.kernel.org/tip/693c819fedcdcabfda7488e2d5e355a84c2fd1b0 Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:25:57 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 15:35:09 -08:00
x86/entry/vdso: Refactor the vdso build
- Separate out the vdso sources into common, vdso32, and vdso64 directories.
- Build the 32- and 64-bit vdsos in their respective subdirectories; this greatly simplifies the build flags handling.
- Unify the mangling of Makefile flags between the 32- and 64-bit vdso code as much as possible; all common rules are put in arch/x86/entry/vdso/common/Makefile.include. The remaining is very simple for 32 bits; the 64-bit one is only slightly more complicated because it contains the x32 generation rule.
- Define __DISABLE_EXPORTS when building the vdso. This need seems to have been masked by different ordering compile flags before.
- Change CONFIG_X86_64 to BUILD_VDSO32_64 in vdso32/system_call.S, to make it compatible with including fake_32bit_build.h.
- The -fcf-protection= option was "leaking" from the kernel build, for reasons that was not clear to me. Furthermore, several distributions ship with it set to a default value other than "-fcf-protection=none". Make it match the configuration options for *user space*.
Note that this patch may seem large, but the vast majority of it is simply code movement.
--- arch/x86/entry/vdso/Makefile | 161 +-------------- arch/x86/entry/vdso/common/Makefile.include | 89 ++++++++- arch/x86/entry/vdso/common/note.S | 18 ++- arch/x86/entry/vdso/common/vclock_gettime.c | 77 +++++++- arch/x86/entry/vdso/common/vdso-layout.lds.S | 101 +++++++++- arch/x86/entry/vdso/common/vgetcpu.c | 22 ++- arch/x86/entry/vdso/vclock_gettime.c | 77 +------- arch/x86/entry/vdso/vdso-layout.lds.S | 101 +--------- arch/x86/entry/vdso/vdso-note.S | 15 +- arch/x86/entry/vdso/vdso.lds.S | 37 +--- arch/x86/entry/vdso/vdso32/Makefile | 24 ++- arch/x86/entry/vdso/vdso32/note.S | 19 +-- arch/x86/entry/vdso/vdso32/system_call.S | 2 +- arch/x86/entry/vdso/vdso32/vclock_gettime.c | 5 +- arch/x86/entry/vdso/vdso32/vdso32.lds.S | 2 +- arch/x86/entry/vdso/vdso32/vgetcpu.c | 4 +- arch/x86/entry/vdso/vdso64/Makefile | 46 ++++- arch/x86/entry/vdso/vdso64/note.S | 1 +- arch/x86/entry/vdso/vdso64/vclock_gettime.c | 1 +- arch/x86/entry/vdso/vdso64/vdso64.lds.S | 37 +++- arch/x86/entry/vdso/vdso64/vdsox32.lds.S | 27 ++- arch/x86/entry/vdso/vdso64/vgetcpu.c | 1 +- arch/x86/entry/vdso/vdso64/vgetrandom-chacha.S | 178 ++++++++++++++++- arch/x86/entry/vdso/vdso64/vgetrandom.c | 15 +- arch/x86/entry/vdso/vdso64/vsgx.S | 150 +++++++++++++- arch/x86/entry/vdso/vdsox32.lds.S | 27 +-- arch/x86/entry/vdso/vgetcpu.c | 22 +-- arch/x86/entry/vdso/vgetrandom-chacha.S | 178 +---------------- arch/x86/entry/vdso/vgetrandom.c | 15 +- arch/x86/entry/vdso/vsgx.S | 150 +------------- 30 files changed, 798 insertions(+), 804 deletions(-) create mode 100644 arch/x86/entry/vdso/common/Makefile.include create mode 100644 arch/x86/entry/vdso/common/note.S create mode 100644 arch/x86/entry/vdso/common/vclock_gettime.c create mode 100644 arch/x86/entry/vdso/common/vdso-layout.lds.S create mode 100644 arch/x86/entry/vdso/common/vgetcpu.c delete mode 100644 arch/x86/entry/vdso/vclock_gettime.c delete mode 100644 arch/x86/entry/vdso/vdso-layout.lds.S delete mode 100644 arch/x86/entry/vdso/vdso-note.S delete mode 100644 arch/x86/entry/vdso/vdso.lds.S create mode 100644 arch/x86/entry/vdso/vdso32/Makefile create mode 100644 arch/x86/entry/vdso/vdso64/Makefile create mode 100644 arch/x86/entry/vdso/vdso64/note.S create mode 100644 arch/x86/entry/vdso/vdso64/vclock_gettime.c create mode 100644 arch/x86/entry/vdso/vdso64/vdso64.lds.S create mode 100644 arch/x86/entry/vdso/vdso64/vdsox32.lds.S create mode 100644 arch/x86/entry/vdso/vdso64/vgetcpu.c create mode 100644 arch/x86/entry/vdso/vdso64/vgetrandom-chacha.S create mode 100644 arch/x86/entry/vdso/vdso64/vgetrandom.c create mode 100644 arch/x86/entry/vdso/vdso64/vsgx.S delete mode 100644 arch/x86/entry/vdso/vdsox32.lds.S delete mode 100644 arch/x86/entry/vdso/vgetcpu.c delete mode 100644 arch/x86/entry/vdso/vgetrandom-chacha.S delete mode 100644 arch/x86/entry/vdso/vgetrandom.c delete mode 100644 arch/x86/entry/vdso/vsgx.S
arch/x86/entry/vdso/Makefile +6 -155
@@ -3,159 +3,10 @@ # Building vDSO images for x86. # -# Include the generic Makefile to check the built vDSO: -include $(srctree)/lib/vdso/Makefile.include +# Regular kernel objects +obj-y := vma.o extable.o +obj-$(CONFIG_COMPAT_32) += vdso32-setup.o -# Files to link into the vDSO: -vobjs-y := vdso-note.o vclock_gettime.o vgetcpu.o vgetrandom.o vgetrandom-chacha.o -vobjs32-y := vdso32/note.o vdso32/system_call.o vdso32/sigreturn.o -vobjs32-y += vdso32/vclock_gettime.o vdso32/vgetcpu.o -vobjs-$(CONFIG_X86_SGX) += vsgx.o - -# Files to link into the kernel: -obj-y += vma.o extable.o - -# vDSO images to build: -obj-$(CONFIG_X86_64) += vdso64-image.o -obj-$(CONFIG_X86_X32_ABI) += vdsox32-image.o -obj-$(CONFIG_COMPAT_32) += vdso32-image.o vdso32-setup.o - -vobjs := $(addprefix $(obj)/, $(vobjs-y)) -vobjs32 := $(addprefix $(obj)/, $(vobjs32-y)) - -$(obj)/vdso.o: $(obj)/vdso.so - -targets += vdso.lds $(vobjs-y) -targets += vdso32/vdso32.lds $(vobjs32-y) - -targets += $(foreach x, 64 x32 32, vdso-image-$(x).c vdso$(x).so vdso$(x).so.dbg) - -CPPFLAGS_vdso.lds += -P -C - -VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 \ - -z max-page-size=4096 - -$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE - $(call if_changed,vdso_and_check) - -VDSO2C = $(objtree)/arch/x86/tools/vdso2c - -quiet_cmd_vdso2c = VDSO2C $@ - cmd_vdso2c = $(VDSO2C) $< $(<:%.dbg=%) $@ - -$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(VDSO2C) FORCE - $(call if_changed,vdso2c) - -# -# Don't omit frame pointers for ease of userspace debugging, but do -# optimize sibling calls. -# -CFL := $(PROFILING) -mcmodel=small -fPIC -O2 -fasynchronous-unwind-tables -m64 \ - $(filter -g%,$(KBUILD_CFLAGS)) -fno-stack-protector \ - -fno-omit-frame-pointer -foptimize-sibling-calls \ - -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO - -ifdef CONFIG_MITIGATION_RETPOLINE -ifneq ($(RETPOLINE_VDSO_CFLAGS),) - CFL += $(RETPOLINE_VDSO_CFLAGS) -endif -endif - -$(vobjs): KBUILD_CFLAGS := $(filter-out $(PADDING_CFLAGS) $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) $(RANDSTRUCT_CFLAGS) $(KSTACK_ERASE_CFLAGS) $(GCC_PLUGINS_CFLAGS) $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS)) $(CFL) -$(vobjs): KBUILD_AFLAGS += -DBUILD_VDSO - -# -# vDSO code runs in userspace and -pg doesn't help with profiling anyway. -# -CFLAGS_REMOVE_vclock_gettime.o = -pg -CFLAGS_REMOVE_vdso32/vclock_gettime.o = -pg -CFLAGS_REMOVE_vgetcpu.o = -pg -CFLAGS_REMOVE_vdso32/vgetcpu.o = -pg -CFLAGS_REMOVE_vsgx.o = -pg -CFLAGS_REMOVE_vgetrandom.o = -pg - -# -# X32 processes use x32 vDSO to access 64bit kernel data. -# -# Build x32 vDSO image: -# 1. Compile x32 vDSO as 64bit. -# 2. Convert object files to x32. -# 3. Build x32 VDSO image with x32 objects, which contains 64bit codes -# so that it can reach 64bit address space with 64bit pointers. -# - -CPPFLAGS_vdsox32.lds = $(CPPFLAGS_vdso.lds) -VDSO_LDFLAGS_vdsox32.lds = -m elf32_x86_64 -soname linux-vdso.so.1 \ - -z max-page-size=4096 - -# x32-rebranded versions -vobjx32s-y := $(vobjs-y:.o=-x32.o) - -# same thing, but in the output directory -vobjx32s := $(addprefix $(obj)/, $(vobjx32s-y)) - -# Convert 64bit object file to x32 for x32 vDSO. -quiet_cmd_x32 = X32 $@ - cmd_x32 = $(OBJCOPY) -O elf32-x86-64 $< $@ - -$(obj)/%-x32.o: $(obj)/%.o FORCE - $(call if_changed,x32) - -targets += vdsox32.lds $(vobjx32s-y) - -$(obj)/%.so: OBJCOPYFLAGS := -S --remove-section __ex_table -$(obj)/%.so: $(obj)/%.so.dbg FORCE - $(call if_changed,objcopy) - -$(obj)/vdsox32.so.dbg: $(obj)/vdsox32.lds $(vobjx32s) FORCE - $(call if_changed,vdso_and_check) - -CPPFLAGS_vdso32/vdso32.lds = $(CPPFLAGS_vdso.lds) -VDSO_LDFLAGS_vdso32.lds = -m elf_i386 -soname linux-gate.so.1 - -KBUILD_AFLAGS_32 := $(filter-out -m64,$(KBUILD_AFLAGS)) -DBUILD_VDSO -$(obj)/vdso32.so.dbg: KBUILD_AFLAGS = $(KBUILD_AFLAGS_32) -$(obj)/vdso32.so.dbg: asflags-$(CONFIG_X86_64) += -m32 - -KBUILD_CFLAGS_32 := $(filter-out -m64,$(KBUILD_CFLAGS)) -KBUILD_CFLAGS_32 := $(filter-out -mcmodel=kernel,$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out -fno-pic,$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out -mfentry,$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(RANDSTRUCT_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(KSTACK_ERASE_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(GCC_PLUGINS_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(CC_FLAGS_CFI),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(PADDING_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 += -m32 -msoft-float -mregparm=0 -fpic -KBUILD_CFLAGS_32 += -fno-stack-protector -KBUILD_CFLAGS_32 += $(call cc-option, -foptimize-sibling-calls) -KBUILD_CFLAGS_32 += -fno-omit-frame-pointer -KBUILD_CFLAGS_32 += -DDISABLE_BRANCH_PROFILING -KBUILD_CFLAGS_32 += -DBUILD_VDSO - -ifdef CONFIG_MITIGATION_RETPOLINE -ifneq ($(RETPOLINE_VDSO_CFLAGS),) - KBUILD_CFLAGS_32 += $(RETPOLINE_VDSO_CFLAGS) -endif -endif - -$(obj)/vdso32.so.dbg: KBUILD_CFLAGS = $(KBUILD_CFLAGS_32) - -$(obj)/vdso32.so.dbg: $(obj)/vdso32/vdso32.lds $(vobjs32) FORCE - $(call if_changed,vdso_and_check) - -# -# The DSO images are built using a special linker script. -# -quiet_cmd_vdso = VDSO $@ - cmd_vdso = $(LD) -o $@ \ - $(VDSO_LDFLAGS) $(VDSO_LDFLAGS_$(filter %.lds,$(^F))) \ - -T $(filter %.lds,$^) $(filter %.o,$^) - -VDSO_LDFLAGS = -shared --hash-style=both --build-id=sha1 --no-undefined \ - $(call ld-option, --eh-frame-hdr) -Bsymbolic -z noexecstack - -quiet_cmd_vdso_and_check = VDSO $@ - cmd_vdso_and_check = $(cmd_vdso); $(cmd_vdso_check) +# vDSO directories +obj-$(CONFIG_X86_64) += vdso64/ +obj-$(CONFIG_COMPAT_32) += vdso32/
arch/x86/entry/vdso/common/Makefile.include new file +89 -0
89 lines not on this page · raw message
arch/x86/entry/vdso/common/note.S new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text. + * Here we can supply some information useful to userland. + */ + +#include <linux/build-salt.h> +#include <linux/version.h> +#include <linux/elfnote.h> + +/* Ideally this would use UTS_NAME, but using a quoted string here + doesn't work. Remember to change this when changing the + kernel's name. */ +ELFNOTE_START(Linux, 0, "a") + .long LINUX_VERSION_CODE +ELFNOTE_END + +BUILD_SALT
arch/x86/entry/vdso/common/vclock_gettime.c new file +77 -0
77 lines not on this page · raw message
arch/x86/entry/vdso/common/vdso-layout.lds.S new file +101 -0
101 lines not on this page · raw message
arch/x86/entry/vdso/common/vgetcpu.c new file +22 -0
@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright 2006 Andi Kleen, SUSE Labs. +* +* Fast user context implementation of getcpu() +*/ + +#include <linux/kernel.h> +#include <linux/getcpu.h> +#include <asm/segment.h> +#include <vdso/processor.h> + +notrace long +__vdso_getcpu (unsigned *cpu,unsigned *node,struct getcpu_cache *unused) +{ +vdso_read_cpunode (cpu, node); + +return 0 ; +} + +long getcpu (unsigned *cpu,unsigned *node,struct getcpu_cache *tcache) + __attribute__((weak,alias ("__vdso_getcpu" )));
arch/x86/entry/vdso/vclock_gettime.c deleted +0 -77
77 lines not on this page · raw message
arch/x86/entry/vdso/vdso-layout.lds.S deleted +0 -101
101 lines not on this page · raw message
arch/x86/entry/vdso/vdso-note.S deleted +0 -15
@@ -1,15 +0,0 @@ -/* - * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text. - * Here we can supply some information useful to userland. - */ - -#include <linux/build-salt.h> -#include <linux/uts.h> -#include <linux/version.h> -#include <linux/elfnote.h> - -ELFNOTE_START(Linux, 0, "a") - .long LINUX_VERSION_CODE -ELFNOTE_END - -BUILD_SALT
arch/x86/entry/vdso/vdso.lds.S deleted +0 -37
@@ -1,37 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * Linker script for 64-bit vDSO. - * We #include the file to define the layout details. - * - * This file defines the version script giving the user-exported symbols in - * the DSO. - */ - -#define BUILD_VDSO64 - -#include "vdso-layout.lds.S" - -/* - * This controls what userland symbols we export from the vDSO. - */ -VERSION { - LINUX_2.6 { - global: - clock_gettime; - __vdso_clock_gettime; - gettimeofday; - __vdso_gettimeofday; - getcpu; - __vdso_getcpu; - time; - __vdso_time; - clock_getres; - __vdso_clock_getres; -#ifdef CONFIG_X86_SGX - __vdso_sgx_enter_enclave; -#endif - getrandom; - __vdso_getrandom; - local: *; - }; -}
arch/x86/entry/vdso/vdso32/Makefile new file +24 -0
@@ -0,0 +1,24 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# 32-bit vDSO images for x86. +# + +# The vDSOs built in this directory +vdsos-y := 32 + +# Files to link into the vDSO: +vobjs-y := note.o vclock_gettime.o vgetcpu.o +vobjs-y += system_call.o sigreturn.o + +# Compilation flags +flags-y := -DBUILD_VDSO32 -m32 -mregparm=0 +flags-$(CONFIG_X86_64) += -include $(src)/fake_32bit_build.h +flags-remove-y := -m64 + +# The location of this include matters! +include $(src)/../common/Makefile.include + +# Linker options for the vdso +VDSO_LDFLAGS_32 := -m elf_i386 -soname linux-gate.so.1 + +$(obj)/vdso32.so.dbg: $(vobjs)
arch/x86/entry/vdso/vdso32/note.S +1 -18
@@ -1,18 +1 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text. - * Here we can supply some information useful to userland. - */ - -#include <linux/build-salt.h> -#include <linux/version.h> -#include <linux/elfnote.h> - -/* Ideally this would use UTS_NAME, but using a quoted string here - doesn't work. Remember to change this when changing the - kernel's name. */ -ELFNOTE_START(Linux, 0, "a") - .long LINUX_VERSION_CODE -ELFNOTE_END - -BUILD_SALT +#include "common/note.S"
arch/x86/entry/vdso/vdso32/system_call.S +1 -1
@@ -52,7 +52,7 @@ __kernel_vsyscall: #define SYSENTER_SEQUENCE "movl %esp, %ebp; sysenter" #define SYSCALL_SEQUENCE "movl %ecx, %ebp; syscall" -#ifdef CONFIG_X86_64 +#ifdef BUILD_VDSO32_64 /* If SYSENTER (Intel) or SYSCALL32 (AMD) is available, use it. */ ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSENTER32, \ SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32
arch/x86/entry/vdso/vdso32/vclock_gettime.c +1 -4
@@ -1,4 +1 @@ -// SPDX-License-Identifier: GPL-2.0 -#define BUILD_VDSO32 -#include "fake_32bit_build.h" -#include "../vclock_gettime.c" +#include "common/vclock_gettime.c"
arch/x86/entry/vdso/vdso32/vdso32.lds.S +1 -1
@@ -11,7 +11,7 @@ #define BUILD_VDSO32 -#include "../vdso-layout.lds.S" +#include "common/vdso-layout.lds.S" /* The ELF entry point can be used to set the AT_SYSINFO value. */ ENTRY(__kernel_vsyscall);
arch/x86/entry/vdso/vdso32/vgetcpu.c +1 -3
@@ -1,3 +1 @@ -// SPDX-License-Identifier: GPL-2.0 -#include "fake_32bit_build.h" -#include "../vgetcpu.c" +#include "common/vgetcpu.c"
arch/x86/entry/vdso/vdso64/Makefile new file +46 -0
46 lines not on this page · raw message
arch/x86/entry/vdso/vdso64/note.S new file +1 -0
@@ -0,0 +1 @@ +#include "common/note.S"
arch/x86/entry/vdso/vdso64/vclock_gettime.c new file +1 -0
@@ -0,0 +1 @@ +#include "common/vclock_gettime.c"
arch/x86/entry/vdso/vdso64/vdso64.lds.S new file +37 -0
@@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Linker script for 64-bit vDSO. + * We #include the file to define the layout details. + * + * This file defines the version script giving the user-exported symbols in + * the DSO. + */ + +#define BUILD_VDSO64 + +#include "common/vdso-layout.lds.S" + +/* + * This controls what userland symbols we export from the vDSO. + */ +VERSION { + LINUX_2.6 { + global: + clock_gettime; + __vdso_clock_gettime; + gettimeofday; + __vdso_gettimeofday; + getcpu; + __vdso_getcpu; + time; + __vdso_time; + clock_getres; + __vdso_clock_getres; +#ifdef CONFIG_X86_SGX + __vdso_sgx_enter_enclave; +#endif + getrandom; + __vdso_getrandom; + local: *; + }; +}
arch/x86/entry/vdso/vdso64/vdsox32.lds.S new file +27 -0
@@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Linker script for x32 vDSO. + * We #include the file to define the layout details. + * + * This file defines the version script giving the user-exported symbols in + * the DSO. + */ + +#define BUILD_VDSOX32 + +#include "common/vdso-layout.lds.S" + +/* + * This controls what userland symbols we export from the vDSO. + */ +VERSION { + LINUX_2.6 { + global: + __vdso_clock_gettime; + __vdso_gettimeofday; + __vdso_getcpu; + __vdso_time; + __vdso_clock_getres; + local: *; + }; +}
arch/x86/entry/vdso/vdso64/vgetcpu.c new file +1 -0
@@ -0,0 +1 @@ +#include "common/vgetcpu.c"
arch/x86/entry/vdso/vdso64/vgetrandom-chacha.S new file +178 -0
178 lines not on this page · raw message
arch/x86/entry/vdso/vdso64/vgetrandom.c new file +15 -0
@@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2022-2024 Jason A. Donenfeld <[email protected]>. All Rights Reserved. +*/ +#include <linux/types.h> + +#include "lib/vdso/getrandom.c" + +ssize_t __vdso_getrandom (void *buffer,size_t len,unsigned int flags,void *opaque_state,size_t opaque_len) +{ +return __cvdso_getrandom (buffer, len, flags, opaque_state, opaque_len); +} + +ssize_t getrandom (void *,size_t ,unsigned int ,void *,size_t ) + __attribute__((weak,alias ("__vdso_getrandom" )));
arch/x86/entry/vdso/vdso64/vsgx.S new file +150 -0
150 lines not on this page · raw message
arch/x86/entry/vdso/vdsox32.lds.S deleted +0 -27
@@ -1,27 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * Linker script for x32 vDSO. - * We #include the file to define the layout details. - * - * This file defines the version script giving the user-exported symbols in - * the DSO. - */ - -#define BUILD_VDSOX32 - -#include "vdso-layout.lds.S" - -/* - * This controls what userland symbols we export from the vDSO. - */ -VERSION { - LINUX_2.6 { - global: - __vdso_clock_gettime; - __vdso_gettimeofday; - __vdso_getcpu; - __vdso_time; - __vdso_clock_getres; - local: *; - }; -}
arch/x86/entry/vdso/vgetcpu.c deleted +0 -22
@@ -1,22 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* -* Copyright 2006 Andi Kleen, SUSE Labs. -* -* Fast user context implementation of getcpu() -*/ - -#include <linux/kernel.h> -#include <linux/getcpu.h> -#include <asm/segment.h> -#include <vdso/processor.h> - -notrace long -__vdso_getcpu (unsigned *cpu,unsigned *node,struct getcpu_cache *unused) -{ -vdso_read_cpunode (cpu, node); - -return 0 ; -} - -long getcpu (unsigned *cpu,unsigned *node,struct getcpu_cache *tcache) - __attribute__((weak,alias ("__vdso_getcpu" )));
arch/x86/entry/vdso/vgetrandom-chacha.S deleted +0 -178
178 lines not on this page · raw message
arch/x86/entry/vdso/vgetrandom.c deleted +0 -15
@@ -1,15 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* -* Copyright (C) 2022-2024 Jason A. Donenfeld <[email protected]>. All Rights Reserved. -*/ -#include <linux/types.h> - -#include "../../../../lib/vdso/getrandom.c" - -ssize_t __vdso_getrandom (void *buffer,size_t len,unsigned int flags,void *opaque_state,size_t opaque_len) -{ -return __cvdso_getrandom (buffer, len, flags, opaque_state, opaque_len); -} - -ssize_t getrandom (void *,size_t ,unsigned int ,void *,size_t ) - __attribute__((weak,alias ("__vdso_getrandom" )));
arch/x86/entry/vdso/vsgx.S deleted +0 -150
150 lines not on this page · raw message
H. Peter Anvin
maintainerPATCH v4 4/10 x86/entry/vdso32: don't rely on int80_landing_pad for adjusting ip
In reply to H. Peter AnvinThere is no fundamental reason to use the int80_landing_pad symbol to adjust ip when moving the vdso. If ip falls within the vdso, and the vdso is moved, we should change the ip accordingly, regardless of mode or location within the vdso. This *currently* can only happen on 32 bits, but there isn't any reason not to do so generically.
Note that if this is ever possible from a vdso-internal call, then the user space stack will also needed to be adjusted (as well as the shadow stack, if enabled.) Fortunately this is not currently the case.
At the moment, we don't even consider other threads when moving the vdso. The assumption is that it is only used by process freeze/thaw for migration, where this is not an issue.
--- arch/x86/entry/vdso/vma.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
arch/x86/entry/vdso/vma.c +6 -10
@@ -65,16 +65,12 @@ static vm_fault_t vdso_fault(const struct vm_special_mapping *sm,static void vdso_fix_landing (const struct vdso_image *image,struct vm_area_struct *new_vma) { -if (in_ia32_syscall () && image == &vdso32_image) { -struct pt_regs *regs =current_pt_regs (); -unsigned long vdso_land = image->sym_int80_landing_pad; -unsigned long old_land_addr = vdso_land + - (unsigned long )current->mm->context.vdso; - -/* Fixing userspace landing - look at do_fast_syscall_32 */ -if (regs->ip == old_land_addr) - regs->ip = new_vma->vm_start + vdso_land; - } +struct pt_regs *regs =current_pt_regs (); +unsigned long ipoffset = regs->ip - + (unsigned long )current->mm->context.vdso; + +if (ipoffset < image->size) + regs->ip = new_vma->vm_start + ipoffset; } static int vdso_mremap(const struct vm_special_mapping *sm,
Signature · 1 line
2.52.0
[tip: x86/entry] x86/entry/vdso32: Don't rely on int80_landing_pad for adjusting ip
The following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: 6e150b71019f386a021004fafea9ef7189bc6aea Gitweb: https://git.kernel.org/tip/6e150b71019f386a021004fafea9ef7189bc6aea Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:25:58 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 16:37:58 -08:00
x86/entry/vdso32: Don't rely on int80_landing_pad for adjusting ip
There is no fundamental reason to use the int80_landing_pad symbol to adjust ip when moving the vdso. If ip falls within the vdso, and the vdso is moved, we should change the ip accordingly, regardless of mode or location within the vdso. This *currently* can only happen on 32 bits, but there isn't any reason not to do so generically.
Note that if this is ever possible from a vdso-internal call, then the user space stack will also needed to be adjusted (as well as the shadow stack, if enabled.) Fortunately this is not currently the case.
At the moment, we don't even consider other threads when moving the vdso. The assumption is that it is only used by process freeze/thaw for migration, where this is not an issue.
--- arch/x86/entry/vdso/vma.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
arch/x86/entry/vdso/vma.c +6 -10
@@ -65,16 +65,12 @@ static vm_fault_t vdso_fault(const struct vm_special_mapping *sm,static void vdso_fix_landing (const struct vdso_image *image,struct vm_area_struct *new_vma) { -if (in_ia32_syscall () && image == &vdso32_image) { -struct pt_regs *regs =current_pt_regs (); -unsigned long vdso_land = image->sym_int80_landing_pad; -unsigned long old_land_addr = vdso_land + - (unsigned long )current->mm->context.vdso; - -/* Fixing userspace landing - look at do_fast_syscall_32 */ -if (regs->ip == old_land_addr) - regs->ip = new_vma->vm_start + vdso_land; - } +struct pt_regs *regs =current_pt_regs (); +unsigned long ipoffset = regs->ip - + (unsigned long )current->mm->context.vdso; + +if (ipoffset < image->size) + regs->ip = new_vma->vm_start + ipoffset; } static int vdso_mremap(const struct vm_special_mapping *sm,
H. Peter Anvin
maintainerPATCH v4 5/10 x86/entry/vdso32: remove SYSCALL_ENTER_KERNEL macro in sigreturn.S
In reply to H. Peter AnvinA macro SYSCALL_ENTER_KERNEL was defined in sigreturn.S, with the ability of overriding it. The override capability, however, is not used anywhere, and the macro name is potentially confusing because it seems to imply that sysenter/syscall could be used here, which is NOT
true: the sigreturn system calls MUST use int $0x80.
--- arch/x86/entry/vdso/vdso32/sigreturn.S | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
arch/x86/entry/vdso/vdso32/sigreturn.S +2 -6
@@ -3,10 +3,6 @@ #include <asm/unistd_32.h> #include <asm/asm-offsets.h> -#ifndef SYSCALL_ENTER_KERNEL -#define SYSCALL_ENTER_KERNEL int $0x80 -#endif - .text .globl __kernel_sigreturn .type __kernel_sigreturn,@function
@@ -16,7 +12,7 @@ __kernel_sigreturn: .LSTART_sigreturn: popl %eax /* XXX does this mean it needs unwind info? */ movl $__NR_sigreturn, %eax - SYSCALL_ENTER_KERNEL + int $0x80 .LEND_sigreturn: SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) nop
@@ -28,7 +24,7 @@ SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) __kernel_rt_sigreturn: .LSTART_rt_sigreturn: movl $__NR_rt_sigreturn, %eax - SYSCALL_ENTER_KERNEL + int $0x80 .LEND_rt_sigreturn: SYM_INNER_LABEL(vdso32_rt_sigreturn_landing_pad, SYM_L_GLOBAL) nop
Signature · 1 line
2.52.0
[tip: x86/entry] x86/entry/vdso32: Remove SYSCALL_ENTER_KERNEL macro in sigreturn.S
The following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: 98d3e996513ad00b7824ea3bece506fc645547dd Gitweb: https://git.kernel.org/tip/98d3e996513ad00b7824ea3bece506fc645547dd Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:25:59 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 16:37:58 -08:00
x86/entry/vdso32: Remove SYSCALL_ENTER_KERNEL macro in sigreturn.S
A macro SYSCALL_ENTER_KERNEL was defined in sigreturn.S, with the ability of overriding it. The override capability, however, is not used anywhere, and the macro name is potentially confusing because it seems to imply that sysenter/syscall could be used here, which is NOT
true: the sigreturn system calls MUST use int $0x80.
--- arch/x86/entry/vdso/vdso32/sigreturn.S | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
arch/x86/entry/vdso/vdso32/sigreturn.S +2 -6
@@ -3,10 +3,6 @@ #include <asm/unistd_32.h> #include <asm/asm-offsets.h> -#ifndef SYSCALL_ENTER_KERNEL -#define SYSCALL_ENTER_KERNEL int $0x80 -#endif - .text .globl __kernel_sigreturn .type __kernel_sigreturn,@function
@@ -16,7 +12,7 @@ __kernel_sigreturn: .LSTART_sigreturn: popl %eax /* XXX does this mean it needs unwind info? */ movl $__NR_sigreturn, %eax - SYSCALL_ENTER_KERNEL + int $0x80 .LEND_sigreturn: SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) nop
@@ -28,7 +24,7 @@ SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) __kernel_rt_sigreturn: .LSTART_rt_sigreturn: movl $__NR_rt_sigreturn, %eax - SYSCALL_ENTER_KERNEL + int $0x80 .LEND_rt_sigreturn: SYM_INNER_LABEL(vdso32_rt_sigreturn_landing_pad, SYM_L_GLOBAL) nop
H. Peter Anvin
maintainerPATCH v4 6/10 x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
In reply to H. Peter AnvinThe vdso32 sigreturn.S contains open-coded DWARF bytecode, which includes a hack for gdb to not try to step back to a previous call instruction when backtracing from a signal handler.
Neither of those are necessary anymore: the backtracing issue is handled by ".cfi_entry simple" and ".cfi_signal_frame", both of which have been supported for a very long time now, which allows the remaining frame to be built using regular .cfi annotations.
Add a few more register offsets to the signal frame just for good measure.
Replace the nop on fallthrough of the system call (which should never, ever happen) with a ud2a trap.
--- arch/x86/entry/vdso/vdso32/sigreturn.S | 146 ++++++------------------- arch/x86/include/asm/dwarf2.h | 1 + arch/x86/kernel/asm-offsets.c | 6 + 3 files changed, 39 insertions(+), 114 deletions(-)
arch/x86/entry/vdso/vdso32/sigreturn.S +32 -114
@@ -1,136 +1,54 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include <linux/linkage.h> #include <asm/unistd_32.h> +#include <asm/dwarf2.h> #include <asm/asm-offsets.h> +.macro STARTPROC_SIGNAL_FRAME sc + CFI_STARTPROC simple + CFI_SIGNAL_FRAME + /* -4 as pretcode has already been popped */ + CFI_DEF_CFA esp, \sc - 4 + CFI_OFFSET eip, IA32_SIGCONTEXT_ip + CFI_OFFSET eax, IA32_SIGCONTEXT_ax + CFI_OFFSET ebx, IA32_SIGCONTEXT_bx + CFI_OFFSET ecx, IA32_SIGCONTEXT_cx + CFI_OFFSET edx, IA32_SIGCONTEXT_dx + CFI_OFFSET esp, IA32_SIGCONTEXT_sp + CFI_OFFSET ebp, IA32_SIGCONTEXT_bp + CFI_OFFSET esi, IA32_SIGCONTEXT_si + CFI_OFFSET edi, IA32_SIGCONTEXT_di + CFI_OFFSET es, IA32_SIGCONTEXT_es + CFI_OFFSET cs, IA32_SIGCONTEXT_cs + CFI_OFFSET ss, IA32_SIGCONTEXT_ss + CFI_OFFSET ds, IA32_SIGCONTEXT_ds + CFI_OFFSET eflags, IA32_SIGCONTEXT_flags +.endm + .text .globl __kernel_sigreturn .type __kernel_sigreturn,@function - nop /* this guy is needed for .LSTARTFDEDLSI1 below (watch for HACK) */ ALIGN __kernel_sigreturn: -.LSTART_sigreturn: - popl %eax /* XXX does this mean it needs unwind info? */ + STARTPROC_SIGNAL_FRAME IA32_SIGFRAME_sigcontext + popl %eax + CFI_ADJUST_CFA_OFFSET -4 movl $__NR_sigreturn, %eax int $0x80 -.LEND_sigreturn: SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) - nop - .size __kernel_sigreturn,.-.LSTART_sigreturn + ud2a + CFI_ENDPROC + .size __kernel_sigreturn,.-__kernel_sigreturn .globl __kernel_rt_sigreturn .type __kernel_rt_sigreturn,@function ALIGN __kernel_rt_sigreturn: -.LSTART_rt_sigreturn: + STARTPROC_SIGNAL_FRAME IA32_RT_SIGFRAME_sigcontext movl $__NR_rt_sigreturn, %eax int $0x80 -.LEND_rt_sigreturn: SYM_INNER_LABEL(vdso32_rt_sigreturn_landing_pad, SYM_L_GLOBAL) - nop - .size __kernel_rt_sigreturn,.-.LSTART_rt_sigreturn - .previous - - .section .eh_frame,"a",@progbits -.LSTARTFRAMEDLSI1: - .long .LENDCIEDLSI1-.LSTARTCIEDLSI1 -.LSTARTCIEDLSI1: - .long 0 /* CIE ID */ - .byte 1 /* Version number */ - .string "zRS" /* NUL-terminated augmentation string */ - .uleb128 1 /* Code alignment factor */ - .sleb128 -4 /* Data alignment factor */ - .byte 8 /* Return address register column */ - .uleb128 1 /* Augmentation value length */ - .byte 0x1b /* DW_EH_PE_pcrel|DW_EH_PE_sdata4. */ - .byte 0 /* DW_CFA_nop */ - .align 4 -.LENDCIEDLSI1: - .long .LENDFDEDLSI1-.LSTARTFDEDLSI1 /* Length FDE */ -.LSTARTFDEDLSI1: - .long .LSTARTFDEDLSI1-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: The dwarf2 unwind routines will subtract 1 from the - return address to get an address in the middle of the - presumed call instruction. Since we didn't get here via - a call, we need to include the nop before the real start - to make up for it. */ - .long .LSTART_sigreturn-1-. /* PC-relative start address */ - .long .LEND_sigreturn-.LSTART_sigreturn+1 - .uleb128 0 /* Augmentation */ - /* What follows are the instructions for the table generation. - We record the locations of each register saved. This is - complicated by the fact that the "CFA" is always assumed to - be the value of the stack pointer in the caller. This means - that we must define the CFA of this body of code to be the - saved value of the stack pointer in the sigcontext. Which - also means that there is no fixed relation to the other - saved registers, which means that we must use DW_CFA_expression - to compute their addresses. It also means that when we - adjust the stack with the popl, we have to do it all over again. */ - -#define do_cfa_expr(offset) \ - .byte 0x0f; /* DW_CFA_def_cfa_expression */ \ - .uleb128 1f-0f; /* length */ \ -0: .byte 0x74; /* DW_OP_breg4 */ \ - .sleb128 offset; /* offset */ \ - .byte 0x06; /* DW_OP_deref */ \ -1: - -#define do_expr(regno, offset) \ - .byte 0x10; /* DW_CFA_expression */ \ - .uleb128 regno; /* regno */ \ - .uleb128 1f-0f; /* length */ \ -0: .byte 0x74; /* DW_OP_breg4 */ \ - .sleb128 offset; /* offset */ \ -1: - - do_cfa_expr(IA32_SIGCONTEXT_sp+4) - do_expr(0, IA32_SIGCONTEXT_ax+4) - do_expr(1, IA32_SIGCONTEXT_cx+4) - do_expr(2, IA32_SIGCONTEXT_dx+4) - do_expr(3, IA32_SIGCONTEXT_bx+4) - do_expr(5, IA32_SIGCONTEXT_bp+4) - do_expr(6, IA32_SIGCONTEXT_si+4) - do_expr(7, IA32_SIGCONTEXT_di+4) - do_expr(8, IA32_SIGCONTEXT_ip+4) - - .byte 0x42 /* DW_CFA_advance_loc 2 -- nop; popl eax. */ - - do_cfa_expr(IA32_SIGCONTEXT_sp) - do_expr(0, IA32_SIGCONTEXT_ax) - do_expr(1, IA32_SIGCONTEXT_cx) - do_expr(2, IA32_SIGCONTEXT_dx) - do_expr(3, IA32_SIGCONTEXT_bx) - do_expr(5, IA32_SIGCONTEXT_bp) - do_expr(6, IA32_SIGCONTEXT_si) - do_expr(7, IA32_SIGCONTEXT_di) - do_expr(8, IA32_SIGCONTEXT_ip) - - .align 4 -.LENDFDEDLSI1: - - .long .LENDFDEDLSI2-.LSTARTFDEDLSI2 /* Length FDE */ -.LSTARTFDEDLSI2: - .long .LSTARTFDEDLSI2-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: See above wrt unwind library assumptions. */ - .long .LSTART_rt_sigreturn-1-. /* PC-relative start address */ - .long .LEND_rt_sigreturn-.LSTART_rt_sigreturn+1 - .uleb128 0 /* Augmentation */ - /* What follows are the instructions for the table generation. - We record the locations of each register saved. This is - slightly less complicated than the above, since we don't - modify the stack pointer in the process. */ - - do_cfa_expr(IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_sp) - do_expr(0, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_ax) - do_expr(1, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_cx) - do_expr(2, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_dx) - do_expr(3, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_bx) - do_expr(5, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_bp) - do_expr(6, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_si) - do_expr(7, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_di) - do_expr(8, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_ip) - - .align 4 -.LENDFDEDLSI2: + ud2a + CFI_ENDPROC + .size __kernel_rt_sigreturn,.-__kernel_rt_sigreturn .previous
arch/x86/include/asm/dwarf2.h +1 -0
@@ -20,6 +20,7 @@#define CFI_RESTORE_STATE .cfi_restore_state#define CFI_UNDEFINED .cfi_undefined#define CFI_ESCAPE .cfi_escape +#define CFI_SIGNAL_FRAME .cfi_signal_frame#ifndef BUILD_VDSO /*
arch/x86/kernel/asm-offsets.c +6 -0
@@ -63,8 +63,14 @@ static void __used common(void)OFFSET (IA32_SIGCONTEXT_bp, sigcontext_32, bp);OFFSET (IA32_SIGCONTEXT_sp, sigcontext_32, sp);OFFSET (IA32_SIGCONTEXT_ip, sigcontext_32, ip); +OFFSET (IA32_SIGCONTEXT_es, sigcontext_32, es); +OFFSET (IA32_SIGCONTEXT_cs, sigcontext_32, cs); +OFFSET (IA32_SIGCONTEXT_ss, sigcontext_32, ss); +OFFSET (IA32_SIGCONTEXT_ds, sigcontext_32, ds); +OFFSET (IA32_SIGCONTEXT_flags, sigcontext_32, flags);BLANK (); +OFFSET (IA32_SIGFRAME_sigcontext, sigframe_ia32, sc);OFFSET (IA32_RT_SIGFRAME_sigcontext, rt_sigframe_ia32, uc.uc_mcontext);#endif
Signature · 1 line
2.52.0
[tip: x86/entry] x86/entry/vdso32: Remove open-coded DWARF in sigreturn.S
The following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: 884961618ee51307cc63ab620a0bdd710fa0b0af Gitweb: https://git.kernel.org/tip/884961618ee51307cc63ab620a0bdd710fa0b0af Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:26:00 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 16:37:58 -08:00
x86/entry/vdso32: Remove open-coded DWARF in sigreturn.S
The vdso32 sigreturn.S contains open-coded DWARF bytecode, which includes a hack for gdb to not try to step back to a previous call instruction when backtracing from a signal handler.
Neither of those are necessary anymore: the backtracing issue is handled by ".cfi_entry simple" and ".cfi_signal_frame", both of which have been supported for a very long time now, which allows the remaining frame to be built using regular .cfi annotations.
Add a few more register offsets to the signal frame just for good measure.
Replace the nop on fallthrough of the system call (which should never, ever happen) with a ud2a trap.
--- arch/x86/entry/vdso/vdso32/sigreturn.S | 146 +++++------------------- arch/x86/include/asm/dwarf2.h | 1 +- arch/x86/kernel/asm-offsets.c | 6 +- 3 files changed, 39 insertions(+), 114 deletions(-)
arch/x86/entry/vdso/vdso32/sigreturn.S +32 -114
@@ -1,136 +1,54 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include <linux/linkage.h> #include <asm/unistd_32.h> +#include <asm/dwarf2.h> #include <asm/asm-offsets.h> +.macro STARTPROC_SIGNAL_FRAME sc + CFI_STARTPROC simple + CFI_SIGNAL_FRAME + /* -4 as pretcode has already been popped */ + CFI_DEF_CFA esp, \sc - 4 + CFI_OFFSET eip, IA32_SIGCONTEXT_ip + CFI_OFFSET eax, IA32_SIGCONTEXT_ax + CFI_OFFSET ebx, IA32_SIGCONTEXT_bx + CFI_OFFSET ecx, IA32_SIGCONTEXT_cx + CFI_OFFSET edx, IA32_SIGCONTEXT_dx + CFI_OFFSET esp, IA32_SIGCONTEXT_sp + CFI_OFFSET ebp, IA32_SIGCONTEXT_bp + CFI_OFFSET esi, IA32_SIGCONTEXT_si + CFI_OFFSET edi, IA32_SIGCONTEXT_di + CFI_OFFSET es, IA32_SIGCONTEXT_es + CFI_OFFSET cs, IA32_SIGCONTEXT_cs + CFI_OFFSET ss, IA32_SIGCONTEXT_ss + CFI_OFFSET ds, IA32_SIGCONTEXT_ds + CFI_OFFSET eflags, IA32_SIGCONTEXT_flags +.endm + .text .globl __kernel_sigreturn .type __kernel_sigreturn,@function - nop /* this guy is needed for .LSTARTFDEDLSI1 below (watch for HACK) */ ALIGN __kernel_sigreturn: -.LSTART_sigreturn: - popl %eax /* XXX does this mean it needs unwind info? */ + STARTPROC_SIGNAL_FRAME IA32_SIGFRAME_sigcontext + popl %eax + CFI_ADJUST_CFA_OFFSET -4 movl $__NR_sigreturn, %eax int $0x80 -.LEND_sigreturn: SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) - nop - .size __kernel_sigreturn,.-.LSTART_sigreturn + ud2a + CFI_ENDPROC + .size __kernel_sigreturn,.-__kernel_sigreturn .globl __kernel_rt_sigreturn .type __kernel_rt_sigreturn,@function ALIGN __kernel_rt_sigreturn: -.LSTART_rt_sigreturn: + STARTPROC_SIGNAL_FRAME IA32_RT_SIGFRAME_sigcontext movl $__NR_rt_sigreturn, %eax int $0x80 -.LEND_rt_sigreturn: SYM_INNER_LABEL(vdso32_rt_sigreturn_landing_pad, SYM_L_GLOBAL) - nop - .size __kernel_rt_sigreturn,.-.LSTART_rt_sigreturn - .previous - - .section .eh_frame,"a",@progbits -.LSTARTFRAMEDLSI1: - .long .LENDCIEDLSI1-.LSTARTCIEDLSI1 -.LSTARTCIEDLSI1: - .long 0 /* CIE ID */ - .byte 1 /* Version number */ - .string "zRS" /* NUL-terminated augmentation string */ - .uleb128 1 /* Code alignment factor */ - .sleb128 -4 /* Data alignment factor */ - .byte 8 /* Return address register column */ - .uleb128 1 /* Augmentation value length */ - .byte 0x1b /* DW_EH_PE_pcrel|DW_EH_PE_sdata4. */ - .byte 0 /* DW_CFA_nop */ - .align 4 -.LENDCIEDLSI1: - .long .LENDFDEDLSI1-.LSTARTFDEDLSI1 /* Length FDE */ -.LSTARTFDEDLSI1: - .long .LSTARTFDEDLSI1-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: The dwarf2 unwind routines will subtract 1 from the - return address to get an address in the middle of the - presumed call instruction. Since we didn't get here via - a call, we need to include the nop before the real start - to make up for it. */ - .long .LSTART_sigreturn-1-. /* PC-relative start address */ - .long .LEND_sigreturn-.LSTART_sigreturn+1 - .uleb128 0 /* Augmentation */ - /* What follows are the instructions for the table generation. - We record the locations of each register saved. This is - complicated by the fact that the "CFA" is always assumed to - be the value of the stack pointer in the caller. This means - that we must define the CFA of this body of code to be the - saved value of the stack pointer in the sigcontext. Which - also means that there is no fixed relation to the other - saved registers, which means that we must use DW_CFA_expression - to compute their addresses. It also means that when we - adjust the stack with the popl, we have to do it all over again. */ - -#define do_cfa_expr(offset) \ - .byte 0x0f; /* DW_CFA_def_cfa_expression */ \ - .uleb128 1f-0f; /* length */ \ -0: .byte 0x74; /* DW_OP_breg4 */ \ - .sleb128 offset; /* offset */ \ - .byte 0x06; /* DW_OP_deref */ \ -1: - -#define do_expr(regno, offset) \ - .byte 0x10; /* DW_CFA_expression */ \ - .uleb128 regno; /* regno */ \ - .uleb128 1f-0f; /* length */ \ -0: .byte 0x74; /* DW_OP_breg4 */ \ - .sleb128 offset; /* offset */ \ -1: - - do_cfa_expr(IA32_SIGCONTEXT_sp+4) - do_expr(0, IA32_SIGCONTEXT_ax+4) - do_expr(1, IA32_SIGCONTEXT_cx+4) - do_expr(2, IA32_SIGCONTEXT_dx+4) - do_expr(3, IA32_SIGCONTEXT_bx+4) - do_expr(5, IA32_SIGCONTEXT_bp+4) - do_expr(6, IA32_SIGCONTEXT_si+4) - do_expr(7, IA32_SIGCONTEXT_di+4) - do_expr(8, IA32_SIGCONTEXT_ip+4) - - .byte 0x42 /* DW_CFA_advance_loc 2 -- nop; popl eax. */ - - do_cfa_expr(IA32_SIGCONTEXT_sp) - do_expr(0, IA32_SIGCONTEXT_ax) - do_expr(1, IA32_SIGCONTEXT_cx) - do_expr(2, IA32_SIGCONTEXT_dx) - do_expr(3, IA32_SIGCONTEXT_bx) - do_expr(5, IA32_SIGCONTEXT_bp) - do_expr(6, IA32_SIGCONTEXT_si) - do_expr(7, IA32_SIGCONTEXT_di) - do_expr(8, IA32_SIGCONTEXT_ip) - - .align 4 -.LENDFDEDLSI1: - - .long .LENDFDEDLSI2-.LSTARTFDEDLSI2 /* Length FDE */ -.LSTARTFDEDLSI2: - .long .LSTARTFDEDLSI2-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: See above wrt unwind library assumptions. */ - .long .LSTART_rt_sigreturn-1-. /* PC-relative start address */ - .long .LEND_rt_sigreturn-.LSTART_rt_sigreturn+1 - .uleb128 0 /* Augmentation */ - /* What follows are the instructions for the table generation. - We record the locations of each register saved. This is - slightly less complicated than the above, since we don't - modify the stack pointer in the process. */ - - do_cfa_expr(IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_sp) - do_expr(0, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_ax) - do_expr(1, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_cx) - do_expr(2, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_dx) - do_expr(3, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_bx) - do_expr(5, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_bp) - do_expr(6, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_si) - do_expr(7, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_di) - do_expr(8, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_ip) - - .align 4 -.LENDFDEDLSI2: + ud2a + CFI_ENDPROC + .size __kernel_rt_sigreturn,.-__kernel_rt_sigreturn .previous
arch/x86/include/asm/dwarf2.h +1 -0
@@ -20,6 +20,7 @@#define CFI_RESTORE_STATE .cfi_restore_state#define CFI_UNDEFINED .cfi_undefined#define CFI_ESCAPE .cfi_escape +#define CFI_SIGNAL_FRAME .cfi_signal_frame#ifndef BUILD_VDSO /*
arch/x86/kernel/asm-offsets.c +6 -0
@@ -63,8 +63,14 @@ static void __used common(void)OFFSET (IA32_SIGCONTEXT_bp, sigcontext_32, bp);OFFSET (IA32_SIGCONTEXT_sp, sigcontext_32, sp);OFFSET (IA32_SIGCONTEXT_ip, sigcontext_32, ip); +OFFSET (IA32_SIGCONTEXT_es, sigcontext_32, es); +OFFSET (IA32_SIGCONTEXT_cs, sigcontext_32, cs); +OFFSET (IA32_SIGCONTEXT_ss, sigcontext_32, ss); +OFFSET (IA32_SIGCONTEXT_ds, sigcontext_32, ds); +OFFSET (IA32_SIGCONTEXT_flags, sigcontext_32, flags);BLANK (); +OFFSET (IA32_SIGFRAME_sigcontext, sigframe_ia32, sc);OFFSET (IA32_RT_SIGFRAME_sigcontext, rt_sigframe_ia32, uc.uc_mcontext);#endif
H. Peter Anvin
maintainerPATCH v4 7/10 x86/entry/vdso: include GNU_PROPERTY and GNU_STACK PHDRs
In reply to H. Peter AnvinCurrently the vdso doesn't include .note.gnu.property or a GNU noexec stack annotation (the -z noexecstack in the linker script is ineffective because we specify PHDRs explicitly.)
The motivation is that the dynamic linker currently do not check these.
However, this is a weak excuse: the vdso*.so are also supposed to be usable at link libraries, and there is no reason why the dynamic linker might not want or need to check these in the future, so add them back in -- it is trivial enough.
Use symbolic constants for the PHDR permission flags.
[ v4: drop unrelated formatting changes ]
--- arch/x86/entry/vdso/common/vdso-layout.lds.S | 38 ++++++++++++-------- 1 file changed, 23 insertions(+), 15 deletions(-)
arch/x86/entry/vdso/common/vdso-layout.lds.S +23 -15
@@ -47,18 +47,18 @@ SECTIONS *(.gnu.linkonce.b.*) } :text - /* - * Discard .note.gnu.property sections which are unused and have - * different alignment requirement from vDSO note sections. - */ - /DISCARD/ : { + .note.gnu.property : { *(.note.gnu.property) - } - .note : { *(.note.*) } :text :note - - .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr - .eh_frame : { KEEP (*(.eh_frame)) } :text + } :text :note :gnu_property + .note : { + *(.note*) + } :text :note + .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr + .eh_frame : { + KEEP (*(.eh_frame)) + *(.eh_frame.*) + } :text /* * Text is well-separated from actual data: there's plenty of
@@ -87,15 +87,23 @@ SECTIONS * Very old versions of ld do not recognize this name token; use the constant. */ #define PT_GNU_EH_FRAME 0x6474e550 +#define PT_GNU_STACK 0x6474e551 +#define PT_GNU_PROPERTY 0x6474e553 /* * We must supply the ELF program headers explicitly to get just one * PT_LOAD segment, and set the flags explicitly to make segments read-only. - */ +*/ +#define PF_R FLAGS(4) +#define PF_RW FLAGS(6) +#define PF_RX FLAGS(5) + PHDRS { - text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */ - dynamic PT_DYNAMIC FLAGS(4); /* PF_R */ - note PT_NOTE FLAGS(4); /* PF_R */ - eh_frame_hdr PT_GNU_EH_FRAME; + text PT_LOAD PF_RX FILEHDR PHDRS; + dynamic PT_DYNAMIC PF_R; + note PT_NOTE PF_R; + eh_frame_hdr PT_GNU_EH_FRAME PF_R; + gnu_stack PT_GNU_STACK PF_RW; + gnu_property PT_GNU_PROPERTY PF_R; }
Signature · 1 line
2.52.0
Re: [PATCH v4 07/10] x86/entry/vdso: include GNU_PROPERTY and GNU_STACK PHDRs
Quoted 56 lines · H. Peter Anvin
Currently the vdso doesn't include .note.gnu.property or a GNU noexec stack annotation (the -z noexecstack in the linker script is ineffective because we specify PHDRs explicitly.)
The motivation is that the dynamic linker currently do not check these.
However, this is a weak excuse: the vdso*.so are also supposed to be usable at link libraries, and there is no reason why the dynamic linker might not want or need to check these in the future, so add them back in -- it is trivial enough.
Use symbolic constants for the PHDR permission flags.
[ v4: drop unrelated formatting changes ]
Signed-off-by H. Peter Anvin (Intel) <[email protected]>--- arch/x86/entry/vdso/common/vdso-layout.lds.S | 38 ++++++++++++-------- 1 file changed, 23 insertions(+), 15 deletions(-)arch/x86/entry/vdso/common/vdso-layout.lds.S +10
-10@@ -47,18 +47,18 @@ SECTIONS *(.gnu.linkonce.b.*) } :text - /* - * Discard .note.gnu.property sections which are unused and have - * different alignment requirement from vDSO note sections. - */ - /DISCARD/ : { + .note.gnu.property : { *(.note.gnu.property) - } - .note : { *(.note.*) } :text :note - - .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr - .eh_frame : { KEEP (*(.eh_frame)) } :text + } :text :note :gnu_property + .note : { + *(.note*) + } :text :note + .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr + .eh_frame : { + KEEP (*(.eh_frame)) + *(.eh_frame.*) + } :text /* * Text is well-separated from actual data: there's plenty of
arch/x86/entry/vdso/common/vdso-layout.lds.S SECTIONS · line 91
Quoted 5 lines
arch/x86/entry/vdso/common/vdso-layout.lds.S +2
-0@@ -87,15 +87,23 @@ SECTIONS * Very old versions of ld do not recognize this name token; use the constant. */ #define PT_GNU_EH_FRAME 0x6474e550 +#define PT_GNU_STACK 0x6474e551 +#define PT_GNU_PROPERTY 0x6474e553
Do we even still support the old linkers that need these constants?
Brian Gerst
Re: [PATCH v4 07/10] x86/entry/vdso: include GNU_PROPERTY and GNU_STACK PHDRs
Quoted 55 lines · Brian Gerst
Quoted 53 lines · H. Peter Anvin
Currently the vdso doesn't include .note.gnu.property or a GNU noexec stack annotation (the -z noexecstack in the linker script is ineffective because we specify PHDRs explicitly.)
The motivation is that the dynamic linker currently do not check these.
However, this is a weak excuse: the vdso*.so are also supposed to be usable at link libraries, and there is no reason why the dynamic linker might not want or need to check these in the future, so add them back in -- it is trivial enough.
Use symbolic constants for the PHDR permission flags.
[ v4: drop unrelated formatting changes ]
Signed-off-by H. Peter Anvin (Intel) <[email protected]>--- arch/x86/entry/vdso/common/vdso-layout.lds.S | 38 ++++++++++++-------- 1 file changed, 23 insertions(+), 15 deletions(-)arch/x86/entry/vdso/common/vdso-layout.lds.S +10
-10@@ -47,18 +47,18 @@ SECTIONS *(.gnu.linkonce.b.*) } :text - /* - * Discard .note.gnu.property sections which are unused and have - * different alignment requirement from vDSO note sections. - */ - /DISCARD/ : { + .note.gnu.property : { *(.note.gnu.property) - } - .note : { *(.note.*) } :text :note - - .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr - .eh_frame : { KEEP (*(.eh_frame)) } :text + } :text :note :gnu_property + .note : { + *(.note*) + } :text :note + .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr + .eh_frame : { + KEEP (*(.eh_frame)) + *(.eh_frame.*) + } :text
Quoted 10 lines
arch/x86/entry/vdso/common/vdso-layout.lds.S SECTIONS · line 91
Quoted 8 lines
arch/x86/entry/vdso/common/vdso-layout.lds.S +2
-0/* * Text is well-separated from actual data: there's plenty of@@ -87,15 +87,23 @@ SECTIONS * Very old versions of ld do not recognize this name token; use the constant. */ #define PT_GNU_EH_FRAME 0x6474e550 +#define PT_GNU_STACK 0x6474e551 +#define PT_GNU_PROPERTY 0x6474e553Do we even still support the old linkers that need these constants?
Digging into the binutils source, PT_GNU_EH_FRAME and PT_GNU_STACK were added to the parser around bintils-2.15. PT_GNU_PROPERTY was added in binutils-2.38, which is newer than the minimum supported version of binutils-2.30.
Probably better to just leave them then.
Brian Gerst
[tip: x86/entry] x86/entry/vdso: Include GNU_PROPERTY and GNU_STACK PHDRs
In reply to H. Peter AnvinThe following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: 8717b02b8c030dc0c4b55781b59e88def0a1a92f Gitweb: https://git.kernel.org/tip/8717b02b8c030dc0c4b55781b59e88def0a1a92f Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:26:01 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 16:37:58 -08:00
x86/entry/vdso: Include GNU_PROPERTY and GNU_STACK PHDRs
Currently the vdso doesn't include .note.gnu.property or a GNU noexec stack annotation (the -z noexecstack in the linker script is ineffective because we specify PHDRs explicitly.)
The motivation is that the dynamic linker currently do not check these.
However, this is a weak excuse: the vdso*.so are also supposed to be usable at link libraries, and there is no reason why the dynamic linker might not want or need to check these in the future, so add them back in -- it is trivial enough.
Use symbolic constants for the PHDR permission flags.
[ v4: drop unrelated formatting changes ]
--- arch/x86/entry/vdso/common/vdso-layout.lds.S | 38 +++++++++++-------- 1 file changed, 23 insertions(+), 15 deletions(-)
arch/x86/entry/vdso/common/vdso-layout.lds.S +23 -15
@@ -47,18 +47,18 @@ SECTIONS *(.gnu.linkonce.b.*) } :text - /* - * Discard .note.gnu.property sections which are unused and have - * different alignment requirement from vDSO note sections. - */ - /DISCARD/ : { + .note.gnu.property : { *(.note.gnu.property) - } - .note : { *(.note.*) } :text :note - - .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr - .eh_frame : { KEEP (*(.eh_frame)) } :text + } :text :note :gnu_property + .note : { + *(.note*) + } :text :note + .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr + .eh_frame : { + KEEP (*(.eh_frame)) + *(.eh_frame.*) + } :text /* * Text is well-separated from actual data: there's plenty of
@@ -87,15 +87,23 @@ SECTIONS * Very old versions of ld do not recognize this name token; use the constant. */ #define PT_GNU_EH_FRAME 0x6474e550 +#define PT_GNU_STACK 0x6474e551 +#define PT_GNU_PROPERTY 0x6474e553 /* * We must supply the ELF program headers explicitly to get just one * PT_LOAD segment, and set the flags explicitly to make segments read-only. - */ +*/ +#define PF_R FLAGS(4) +#define PF_RW FLAGS(6) +#define PF_RX FLAGS(5) + PHDRS { - text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */ - dynamic PT_DYNAMIC FLAGS(4); /* PF_R */ - note PT_NOTE FLAGS(4); /* PF_R */ - eh_frame_hdr PT_GNU_EH_FRAME; + text PT_LOAD PF_RX FILEHDR PHDRS; + dynamic PT_DYNAMIC PF_R; + note PT_NOTE PF_R; + eh_frame_hdr PT_GNU_EH_FRAME PF_R; + gnu_stack PT_GNU_STACK PF_RW; + gnu_property PT_GNU_PROPERTY PF_R; }
H. Peter Anvin
maintainerPATCH v4 8/10 x86/vdso: abstract out vdso system call internals
In reply to H. Peter AnvinAbstract out the calling of true system calls from the vdso into macros.
It has been a very long time since gcc did not allow %ebx or %ebp in inline asm in 32-bit PIC mode; remove the corresponding hacks.
Remove the use of memory output constraints in gettimeofday.h in favor of "memory" clobbers. The resulting code is identical for the current use cases, as the system call is usually a terminal fallback anyway, and it merely complicates the macroization.
This patch adds only a handful of more lines of code than it removes, and in fact could be made substantially smaller by removing the macros for the argument counts that aren't currently used, however, it seems better to be general from the start.
[ v3: remove stray comment from prototyping; remove VDSO_SYSCALL6()
since it would require special handling on 32 bits and is currently unused. (Uros Biszjak)
Indent nested preprocessor directives. ]
--- arch/x86/include/asm/vdso/gettimeofday.h | 108 ++--------------------- arch/x86/include/asm/vdso/sys_call.h | 103 +++++++++++++++++++++ 2 files changed, 111 insertions(+), 100 deletions(-) create mode 100644 arch/x86/include/asm/vdso/sys_call.h
arch/x86/include/asm/vdso/gettimeofday.h +8 -100
@@ -18,6 +18,7 @@#include <asm/msr.h> #include <asm/pvclock.h> #include <clocksource/hyperv_timer.h> +#include <asm/vdso/sys_call.h> #define VDSO_HAS_TIME 1
@@ -53,130 +54,37 @@ extern struct ms_hyperv_tsc_page hvclock_page __attribute__((visibility("hidden")));#endif -#ifndef BUILD_VDSO32 -static __always_inline long clock_gettime_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) { -long ret; - - asm ("syscall" :"=a" (ret),"=m" (*_ts) : -"0" (__NR_clock_gettime),"D" (_clkid),"S" (_ts) : -"rcx" ,"r11" ); - -return ret; +return VDSO_SYSCALL2 (clock_gettime,64 ,_clkid,_ts); }static __always_inline long gettimeofday_fallback (struct __kernel_old_timeval *_tv,struct timezone *_tz) { -long ret; - - asm("syscall" :"=a" (ret) : -"0" (__NR_gettimeofday),"D" (_tv),"S" (_tz) :"memory" ); - -return ret; + return VDSO_SYSCALL2(gettimeofday,,_tv,_tz); }static __always_inline long clock_getres_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) { -long ret; - - asm ("syscall" :"=a" (ret),"=m" (*_ts) : -"0" (__NR_clock_getres),"D" (_clkid),"S" (_ts) : -"rcx" ,"r11" ); - -return ret; +return VDSO_SYSCALL2 (clock_getres,_time64,_clkid,_ts); } -#else - -static __always_inline -long clock_gettime_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) -{ -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_gettime64), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} +#ifndef CONFIG_X86_64 static __always_inline long clock_gettime32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) { -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_gettime), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} - -static __always_inline -long gettimeofday_fallback (struct __kernel_old_timeval *_tv, -struct timezone *_tz) -{ -long ret; - - asm( -"mov %%ebx, %%edx \n" -"mov %2, %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret) - :"0" (__NR_gettimeofday),"g" (_tv),"c" (_tz) - :"memory" ,"edx" ); - -return ret; + return VDSO_SYSCALL2(clock_gettime,,_clkid,_ts); }static __always_inline long -clock_getres_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) -{ -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_getres_time64), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} - -static __always_inline -long clock_getres32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) +clock_getres32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) { -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_getres), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; + return VDSO_SYSCALL2(clock_getres,,_clkid,_ts); }#endif
arch/x86/include/asm/vdso/sys_call.h new file +103 -0
103 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v4 08/10] x86/vdso: abstract out vdso system call internals
Quoted 17 lines · H. Peter Anvin
Abstract out the calling of true system calls from the vdso into macros.
It has been a very long time since gcc did not allow %ebx or %ebp in inline asm in 32-bit PIC mode; remove the corresponding hacks.
Remove the use of memory output constraints in
gettimeofday.hin favor of "memory" clobbers. The resulting code is identical for the current use cases, as the system call is usually a terminal fallback anyway, and it merely complicates the macroization.This patch adds only a handful of more lines of code than it removes, and in fact could be made substantially smaller by removing the macros for the argument counts that aren't currently used, however, it seems better to be general from the start.
Quoted 7 lines
[ v3: remove stray comment from prototyping; remove
VDSO_SYSCALL6()
since it would require special handling on 32 bits and is currently unused. (Uros Biszjak)Indent nested preprocessor directives. ]
Signed-off-by H. Peter Anvin (Intel) <[email protected]>
Quoted 268 lines
--- arch/x86/include/asm/vdso/gettimeofday.h | 108 ++--------------------- arch/x86/include/asm/vdso/sys_call.h | 103 +++++++++++++++++++++ 2 files changed, 111 insertions(+), 100 deletions(-) create mode 100644 arch/x86/include/asm/vdso/sys_call.harch/x86/include/asm/vdso/gettimeofday.h +8
-100@@ -18,6 +18,7 @@#include <asm/msr.h> #include <asm/pvclock.h> #include <clocksource/hyperv_timer.h> +#include <asm/vdso/sys_call.h> #define VDSO_HAS_TIME 1@@ -53,130 +54,37 @@ extern struct ms_hyperv_tsc_page hvclock_page __attribute__((visibility("hidden")));#endif -#ifndef BUILD_VDSO32 -static __always_inline long clock_gettime_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) { -long ret; - - asm ("syscall" :"=a" (ret),"=m" (*_ts) : -"0" (__NR_clock_gettime),"D" (_clkid),"S" (_ts) : -"rcx" ,"r11" ); - -return ret; +return VDSO_SYSCALL2 (clock_gettime,64 ,_clkid,_ts); }static __always_inline long gettimeofday_fallback (struct __kernel_old_timeval *_tv,struct timezone *_tz) { -long ret; - - asm("syscall" :"=a" (ret) : -"0" (__NR_gettimeofday),"D" (_tv),"S" (_tz) :"memory" ); - -return ret; + return VDSO_SYSCALL2(gettimeofday,,_tv,_tz); }static __always_inline long clock_getres_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) { -long ret; - - asm ("syscall" :"=a" (ret),"=m" (*_ts) : -"0" (__NR_clock_getres),"D" (_clkid),"S" (_ts) : -"rcx" ,"r11" ); - -return ret; +return VDSO_SYSCALL2 (clock_getres,_time64,_clkid,_ts); } -#else - -static __always_inline -long clock_gettime_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) -{ -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_gettime64), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} +#ifndef CONFIG_X86_64 static __always_inline long clock_gettime32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) { -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_gettime), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} - -static __always_inline -long gettimeofday_fallback (struct __kernel_old_timeval *_tv, -struct timezone *_tz) -{ -long ret; - - asm( -"mov %%ebx, %%edx \n" -"mov %2, %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret) - :"0" (__NR_gettimeofday),"g" (_tv),"c" (_tz) - :"memory" ,"edx" ); - -return ret; + return VDSO_SYSCALL2(clock_gettime,,_clkid,_ts); }static __always_inline long -clock_getres_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) -{ -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_getres_time64), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} - -static __always_inline -long clock_getres32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) +clock_getres32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) { -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_getres), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; + return VDSO_SYSCALL2(clock_getres,,_clkid,_ts); }#endif arch/x86/include/asm/vdso/sys_call.h new file +103
-1@@ -0,0 +1,103 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Macros for issuing an inline system call from the vDSO. +*/ + +#ifndef X86_ASM_VDSO_SYS_CALL_H +#define X86_ASM_VDSO_SYS_CALL_H + +#include <linux/compiler.h> +#include <asm/cpufeatures.h> +#include <asm/alternative.h> + +#ifdef CONFIG_X86_64 +# define __sys_instr "syscall" +# define __sys_clobber "rcx", "r11", "memory" +# define __sys_nr (x,y) __NR_ ## x +# define __sys_reg1 "rdi" +# define __sys_reg2 "rsi" +# define __sys_reg3 "rdx" +# define __sys_reg4 "r10" +# define __sys_reg5 "r8" +#else +# define __sys_instr "call __kernel_vsyscall" +# define __sys_clobber "memory" +# define __sys_nr (x,y) __NR_ ## x ## y +# define __sys_reg1 "ebx" +# define __sys_reg2 "ecx" +# define __sys_reg3 "edx" +# define __sys_reg4 "esi" +# define __sys_reg5 "edi" +#endif + +/* +* Example usage: +* +* result = VDSO_SYSCALL3(foo,64,x,y,z); +* +* ... calls foo(x,y,z) on 64 bits, and foo64(x,y,z) on 32 bits. +* +* VDSO_SYSCALL6() is currently missing, because it would require +* special handling for %ebp on 32 bits when the vdso is compiled with +* frame pointers enabled (the default on 32 bits.) Add it as a special +* case when and if it becomes necessary. +*/ +#define _VDSO_SYSCALL (name,suf32,...) \ + ({ \ + long _sys_num_ret = __sys_nr(name,suf32); \ + asm_inline volatile( \ + __sys_instr \ + : "+a" (_sys_num_ret) \ + : __VA_ARGS__ \ + : __sys_clobber); \ + _sys_num_ret; \ + }) + +#define VDSO_SYSCALL0 (name,suf32) \ + _VDSO_SYSCALL(name,suf32) +#define VDSO_SYSCALL1 (name,suf32,a1) \ + ({ \ + register long _sys_arg1 asm(__sys_reg1) = (long)(a1); \ + _VDSO_SYSCALL(name,suf32, \ + "r" (_sys_arg1)); \ + }) +#define VDSO_SYSCALL2 (name,suf32,a1,a2) \ + ({ \ + register long _sys_arg1 asm(__sys_reg1) = (long)(a1); \ + register long _sys_arg2 asm(__sys_reg2) = (long)(a2); \ + _VDSO_SYSCALL(name,suf32, \ + "r" (_sys_arg1), "r" (_sys_arg2)); \ + }) +#define VDSO_SYSCALL3 (name,suf32,a1,a2,a3) \ + ({ \ + register long _sys_arg1 asm(__sys_reg1) = (long)(a1); \ + register long _sys_arg2 asm(__sys_reg2) = (long)(a2); \ + register long _sys_arg3 asm(__sys_reg3) = (long)(a3); \ + _VDSO_SYSCALL(name,suf32, \ + "r" (_sys_arg1), "r" (_sys_arg2), \ + "r" (_sys_arg3)); \ + }) +#define VDSO_SYSCALL4 (name,suf32,a1,a2,a3,a4) \ + ({ \ + register long _sys_arg1 asm(__sys_reg1) = (long)(a1); \ + register long _sys_arg2 asm(__sys_reg2) = (long)(a2); \ + register long _sys_arg3 asm(__sys_reg3) = (long)(a3); \ + register long _sys_arg4 asm(__sys_reg4) = (long)(a4); \ + _VDSO_SYSCALL(name,suf32, \ + "r" (_sys_arg1), "r" (_sys_arg2), \ + "r" (_sys_arg3), "r" (_sys_arg4)); \ + }) +#define VDSO_SYSCALL5 (name,suf32,a1,a2,a3,a4,a5) \ + ({ \ + register long _sys_arg1 asm(__sys_reg1) = (long)(a1); \ + register long _sys_arg2 asm(__sys_reg2) = (long)(a2); \ + register long _sys_arg3 asm(__sys_reg3) = (long)(a3); \ + register long _sys_arg4 asm(__sys_reg4) = (long)(a4); \ + register long _sys_arg5 asm(__sys_reg5) = (long)(a5); \ + _VDSO_SYSCALL(name,suf32, \ + "r" (_sys_arg1), "r" (_sys_arg2), \ + "r" (_sys_arg3), "r" (_sys_arg4), \ + "r" (_sys_arg5)); \ + }) + +#endif /* X86_VDSO_SYS_CALL_H */ --2.52.0
[tip: x86/entry] x86/vdso: Abstract out vdso system call internals
In reply to H. Peter AnvinThe following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: a0636d4c3ad0da0cd6069eb6fef5d2b7d3449378 Gitweb: https://git.kernel.org/tip/a0636d4c3ad0da0cd6069eb6fef5d2b7d3449378 Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:26:02 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 16:37:58 -08:00
x86/vdso: Abstract out vdso system call internals
Abstract out the calling of true system calls from the vdso into macros.
It has been a very long time since gcc did not allow %ebx or %ebp in inline asm in 32-bit PIC mode; remove the corresponding hacks.
Remove the use of memory output constraints in gettimeofday.h in favor of "memory" clobbers. The resulting code is identical for the current use cases, as the system call is usually a terminal fallback anyway, and it merely complicates the macroization.
This patch adds only a handful of more lines of code than it removes, and in fact could be made substantially smaller by removing the macros for the argument counts that aren't currently used, however, it seems better to be general from the start.
[ v3: remove stray comment from prototyping; remove VDSO_SYSCALL6()
since it would require special handling on 32 bits and is currently unused. (Uros Biszjak)
Indent nested preprocessor directives. ]
--- arch/x86/include/asm/vdso/gettimeofday.h | 108 +--------------------- arch/x86/include/asm/vdso/sys_call.h | 103 +++++++++++++++++++++- 2 files changed, 111 insertions(+), 100 deletions(-) create mode 100644 arch/x86/include/asm/vdso/sys_call.h
arch/x86/include/asm/vdso/gettimeofday.h +8 -100
@@ -18,6 +18,7 @@#include <asm/msr.h> #include <asm/pvclock.h> #include <clocksource/hyperv_timer.h> +#include <asm/vdso/sys_call.h> #define VDSO_HAS_TIME 1
@@ -53,130 +54,37 @@ extern struct ms_hyperv_tsc_page hvclock_page __attribute__((visibility("hidden")));#endif -#ifndef BUILD_VDSO32 -static __always_inline long clock_gettime_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) { -long ret; - - asm ("syscall" :"=a" (ret),"=m" (*_ts) : -"0" (__NR_clock_gettime),"D" (_clkid),"S" (_ts) : -"rcx" ,"r11" ); - -return ret; +return VDSO_SYSCALL2 (clock_gettime,64 ,_clkid,_ts); }static __always_inline long gettimeofday_fallback (struct __kernel_old_timeval *_tv,struct timezone *_tz) { -long ret; - - asm("syscall" :"=a" (ret) : -"0" (__NR_gettimeofday),"D" (_tv),"S" (_tz) :"memory" ); - -return ret; + return VDSO_SYSCALL2(gettimeofday,,_tv,_tz); }static __always_inline long clock_getres_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) { -long ret; - - asm ("syscall" :"=a" (ret),"=m" (*_ts) : -"0" (__NR_clock_getres),"D" (_clkid),"S" (_ts) : -"rcx" ,"r11" ); - -return ret; +return VDSO_SYSCALL2 (clock_getres,_time64,_clkid,_ts); } -#else - -static __always_inline -long clock_gettime_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) -{ -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_gettime64), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} +#ifndef CONFIG_X86_64 static __always_inline long clock_gettime32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) { -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_gettime), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} - -static __always_inline -long gettimeofday_fallback (struct __kernel_old_timeval *_tv, -struct timezone *_tz) -{ -long ret; - - asm( -"mov %%ebx, %%edx \n" -"mov %2, %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret) - :"0" (__NR_gettimeofday),"g" (_tv),"c" (_tz) - :"memory" ,"edx" ); - -return ret; + return VDSO_SYSCALL2(clock_gettime,,_clkid,_ts); }static __always_inline long -clock_getres_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) -{ -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_getres_time64), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} - -static __always_inline -long clock_getres32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) +clock_getres32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) { -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_getres), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; + return VDSO_SYSCALL2(clock_getres,,_clkid,_ts); }#endif
arch/x86/include/asm/vdso/sys_call.h new file +103 -0
103 lines not on this page · raw message
H. Peter Anvin
maintainerPATCH v4 9/10 x86/cpufeature: replace X86_FEATURE_SYSENTER32 with X86_FEATURE_SYSFAST32
In reply to H. Peter AnvinIn most cases, the use of "fast 32-bit system call" depends either on X86_FEATURE_SEP or X86_FEATURE_SYSENTER32 || X86_FEATURE_SYSCALL32. However, nearly all the logic for both is identical.
Define X86_FEATURE_SYSFAST32 which indicates that *either* SYSENTER32 or SYSCALL32 should be used, for either 32- or 64-bit kernels. This defaults to SYSENTER; use SYSCALL if the SYSCALL32 bit is also set.
As this removes ALL existing uses of X86_FEATURE_SYSENTER32, which is a kernel-only synthetic feature bit, simply remove it and replace it with X86_FEATURE_SYSFAST32.
This leaves an unused alternative for a true 32-bit kernel, but that should really not matter in any way.
The clearing of X86_FEATURE_SYSCALL32 can be removed once the patches for automatically clearing disabled features has been merged.
--- arch/x86/Kconfig.cpufeatures | 8 +++++++ arch/x86/entry/vdso/vdso32/system_call.S | 8 ++----- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/centaur.c | 3 --- arch/x86/kernel/cpu/common.c | 8 +++++++ arch/x86/kernel/cpu/intel.c | 4 +--- arch/x86/kernel/cpu/zhaoxin.c | 4 +--- arch/x86/kernel/fred.c | 2 +- arch/x86/xen/setup.c | 28 +++++++++++++++--------- arch/x86/xen/smp_pv.c | 5 ++--- arch/x86/xen/xen-ops.h | 1 - 11 files changed, 42 insertions(+), 31 deletions(-)
arch/x86/Kconfig.cpufeatures +8 -0
@@ -56,6 +56,10 @@ config X86_REQUIRED_FEATURE_MOVBE def_bool y depends on MATOM +config X86_REQUIRED_FEATURE_SYSFAST32 + def_bool y + depends on X86_64 && !X86_FRED + config X86_REQUIRED_FEATURE_CPUID def_bool y depends on X86_64
@@ -120,6 +124,10 @@ config X86_DISABLED_FEATURE_CENTAUR_MCR def_bool y depends on X86_64 +config X86_DISABLED_FEATURE_SYSCALL32 + def_bool y + depends on !X86_64 + config X86_DISABLED_FEATURE_PCID def_bool y depends on !X86_64
arch/x86/entry/vdso/vdso32/system_call.S +2 -6
@@ -52,13 +52,9 @@ __kernel_vsyscall: #define SYSENTER_SEQUENCE "movl %esp, %ebp; sysenter" #define SYSCALL_SEQUENCE "movl %ecx, %ebp; syscall" -#ifdef BUILD_VDSO32_64 /* If SYSENTER (Intel) or SYSCALL32 (AMD) is available, use it. */ - ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSENTER32, \ - SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 -#else - ALTERNATIVE "", SYSENTER_SEQUENCE, X86_FEATURE_SEP -#endif + ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSFAST32, \ + SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 /* Enter using int $0x80 */ int $0x80
arch/x86/include/asm/cpufeatures.h +1 -1
@@ -84,7 +84,7 @@#define X86_FEATURE_PEBS ( 3*32+12)/* "pebs" Precise-Event Based Sampling */ #define X86_FEATURE_BTS ( 3*32+13)/* "bts" Branch Trace Store */ #define X86_FEATURE_SYSCALL32 ( 3*32+14)/* syscall in IA32 userspace */ -#define X86_FEATURE_SYSENTER32 ( 3*32+15)/* sysenter in IA32 userspace */ +#define X86_FEATURE_SYSFAST32 ( 3*32+15)/* sysenter/syscall in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16)/* "rep_good" REP microcode works well */ #define X86_FEATURE_AMD_LBR_V2 ( 3*32+17)/* "amd_lbr_v2" AMD Last Branch Record Extension Version 2 */ #define X86_FEATURE_CLEAR_CPU_BUF ( 3*32+18)/* Clear CPU buffers using VERW */
arch/x86/kernel/cpu/centaur.c +0 -3
@@ -102,9 +102,6 @@ static void early_init_centaur(struct cpuinfo_x86 *c) (c->x86 >= 7))set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC ); -#ifdef CONFIG_X86_64 -set_cpu_cap (c,X86_FEATURE_SYSENTER32 ); -#endif if (c->x86_power & (1 <<8 )) {set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC );set_cpu_cap (c,X86_FEATURE_NONSTOP_TSC );
arch/x86/kernel/cpu/common.c +8 -0
@@ -1068,6 +1068,9 @@ void get_cpu_cap(struct cpuinfo_x86 *c)init_scattered_cpuid_features (c);init_speculation_control (c); +if (IS_ENABLED (CONFIG_X86_64 ) ||cpu_has (c,X86_FEATURE_SEP )) +set_cpu_cap (c,X86_FEATURE_SYSFAST32 ); +/* * Clear/Set all flags overridden by options, after probe. * This needs to happen each time we re-probe, which may happen
@@ -1813,6 +1816,11 @@ static void __init early_identify_cpu(struct cpuinfo_x86 *c)* that it can't be enabled in 32-bit mode. */ setup_clear_cpu_cap (X86_FEATURE_PCID ); + +/* +* Never use SYSCALL on a 32-bit kernel +*/ +setup_clear_cpu_cap (X86_FEATURE_SYSCALL32 );#endif /*
arch/x86/kernel/cpu/intel.c +1 -3
@@ -236,9 +236,7 @@ static void early_init_intel(struct cpuinfo_x86 *c)clear_cpu_cap (c,X86_FEATURE_PSE ); } -#ifdef CONFIG_X86_64 -set_cpu_cap (c,X86_FEATURE_SYSENTER32 ); -#else +#ifndef CONFIG_X86_64 /* Netburst reports 64 bytes clflush size, but does IO in 128 bytes */ if (c->x86 ==15 && c->x86_cache_alignment ==64 ) c->x86_cache_alignment =128 ;
arch/x86/kernel/cpu/zhaoxin.c +1 -3
@@ -59,9 +59,7 @@ static void early_init_zhaoxin(struct cpuinfo_x86 *c) {if (c->x86 >=0x6 )set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC ); -#ifdef CONFIG_X86_64 -set_cpu_cap (c,X86_FEATURE_SYSENTER32 ); -#endif +if (c->x86_power & (1 <<8 )) {set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC );set_cpu_cap (c,X86_FEATURE_NONSTOP_TSC );
arch/x86/kernel/fred.c +1 -1
@@ -68,7 +68,7 @@ void cpu_init_fred_exceptions(void)idt_invalidate ();/* Use int $0x80 for 32-bit system calls in FRED mode */ -setup_clear_cpu_cap (X86_FEATURE_SYSENTER32 ); +setup_clear_cpu_cap (X86_FEATURE_SYSFAST32 );setup_clear_cpu_cap (X86_FEATURE_SYSCALL32 ); }
arch/x86/xen/setup.c +18 -10
@@ -990,13 +990,6 @@ static int register_callback(unsigned type, const void *func)return HYPERVISOR_callback_op (CALLBACKOP_register, &callback); } -void xen_enable_sysenter (void ) -{ -if (cpu_feature_enabled (X86_FEATURE_SYSENTER32 ) && -register_callback (CALLBACKTYPE_sysenter, xen_entry_SYSENTER_compat)) -setup_clear_cpu_cap (X86_FEATURE_SYSENTER32 ); -} -void xen_enable_syscall (void ) {int ret;
@@ -1008,11 +1001,27 @@ void xen_enable_syscall(void)mechanism for syscalls. */ } -if (cpu_feature_enabled (X86_FEATURE_SYSCALL32 ) && -register_callback (CALLBACKTYPE_syscall32, xen_entry_SYSCALL_compat)) +if (!cpu_feature_enabled (X86_FEATURE_SYSFAST32 )) +return ; + +if (cpu_feature_enabled (X86_FEATURE_SYSCALL32 )) { +/* Use SYSCALL32 */ + ret =register_callback (CALLBACKTYPE_syscall32, + xen_entry_SYSCALL_compat); + + }else { +/* Use SYSENTER32 */ + ret =register_callback (CALLBACKTYPE_sysenter, + xen_entry_SYSENTER_compat); + } + +if (ret) {setup_clear_cpu_cap (X86_FEATURE_SYSCALL32 ); +setup_clear_cpu_cap (X86_FEATURE_SYSFAST32 ); + } } +static void __init xen_pvmmu_arch_setup (void ) {HYPERVISOR_vm_assist (VMASST_CMD_enable, VMASST_TYPE_writable_pagetables);
@@ -1022,7 +1031,6 @@ static void __init xen_pvmmu_arch_setup(void) register_callback(CALLBACKTYPE_failsafe, xen_failsafe_callback))BUG (); -xen_enable_sysenter ();xen_enable_syscall (); }
arch/x86/xen/smp_pv.c +2 -3
@@ -65,10 +65,9 @@ static void cpu_bringup(void)touch_softlockup_watchdog ();/* PVH runs in ring 0 and allows us to do native syscalls. Yay! */ -if (!xen_feature (XENFEAT_supervisor_mode_kernel)) { -xen_enable_sysenter (); +if (!xen_feature (XENFEAT_supervisor_mode_kernel))xen_enable_syscall (); - } + cpu =smp_processor_id ();identify_secondary_cpu (cpu);set_cpu_sibling_map (cpu);
arch/x86/xen/xen-ops.h +0 -1
@@ -60,7 +60,6 @@ phys_addr_t __init xen_find_free_area(phys_addr_t size);char *__init xen_memory_setup (void );void __init xen_arch_setup (void );void xen_banner (void ); -void xen_enable_sysenter (void );void xen_enable_syscall (void );void xen_vcpu_restore (void );
Signature · 1 line
2.52.0
[tip: x86/entry] x86/cpufeature: Replace X86_FEATURE_SYSENTER32 with X86_FEATURE_SYSFAST32
The following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: f49ecf5e110ab0ed255ddea5e321689faf4e50e6 Gitweb: https://git.kernel.org/tip/f49ecf5e110ab0ed255ddea5e321689faf4e50e6 Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:26:03 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 16:37:58 -08:00
x86/cpufeature: Replace X86_FEATURE_SYSENTER32 with X86_FEATURE_SYSFAST32
In most cases, the use of "fast 32-bit system call" depends either on X86_FEATURE_SEP or X86_FEATURE_SYSENTER32 || X86_FEATURE_SYSCALL32. However, nearly all the logic for both is identical.
Define X86_FEATURE_SYSFAST32 which indicates that *either* SYSENTER32 or SYSCALL32 should be used, for either 32- or 64-bit kernels. This defaults to SYSENTER; use SYSCALL if the SYSCALL32 bit is also set.
As this removes ALL existing uses of X86_FEATURE_SYSENTER32, which is a kernel-only synthetic feature bit, simply remove it and replace it with X86_FEATURE_SYSFAST32.
This leaves an unused alternative for a true 32-bit kernel, but that should really not matter in any way.
The clearing of X86_FEATURE_SYSCALL32 can be removed once the patches for automatically clearing disabled features has been merged.
--- arch/x86/Kconfig.cpufeatures | 8 +++++++- arch/x86/entry/vdso/vdso32/system_call.S | 8 +------ arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/centaur.c | 3 +-- arch/x86/kernel/cpu/common.c | 8 +++++++- arch/x86/kernel/cpu/intel.c | 4 +--- arch/x86/kernel/cpu/zhaoxin.c | 4 +--- arch/x86/kernel/fred.c | 2 +- arch/x86/xen/setup.c | 28 ++++++++++++++--------- arch/x86/xen/smp_pv.c | 5 +--- arch/x86/xen/xen-ops.h | 1 +- 11 files changed, 42 insertions(+), 31 deletions(-)
arch/x86/Kconfig.cpufeatures +8 -0
@@ -56,6 +56,10 @@ config X86_REQUIRED_FEATURE_MOVBE def_bool y depends on MATOM +config X86_REQUIRED_FEATURE_SYSFAST32 + def_bool y + depends on X86_64 && !X86_FRED + config X86_REQUIRED_FEATURE_CPUID def_bool y depends on X86_64
@@ -120,6 +124,10 @@ config X86_DISABLED_FEATURE_CENTAUR_MCR def_bool y depends on X86_64 +config X86_DISABLED_FEATURE_SYSCALL32 + def_bool y + depends on !X86_64 + config X86_DISABLED_FEATURE_PCID def_bool y depends on !X86_64
arch/x86/entry/vdso/vdso32/system_call.S +2 -6
@@ -52,13 +52,9 @@ __kernel_vsyscall: #define SYSENTER_SEQUENCE "movl %esp, %ebp; sysenter" #define SYSCALL_SEQUENCE "movl %ecx, %ebp; syscall" -#ifdef BUILD_VDSO32_64 /* If SYSENTER (Intel) or SYSCALL32 (AMD) is available, use it. */ - ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSENTER32, \ - SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 -#else - ALTERNATIVE "", SYSENTER_SEQUENCE, X86_FEATURE_SEP -#endif + ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSFAST32, \ + SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 /* Enter using int $0x80 */ int $0x80
arch/x86/include/asm/cpufeatures.h +1 -1
@@ -84,7 +84,7 @@#define X86_FEATURE_PEBS ( 3*32+12)/* "pebs" Precise-Event Based Sampling */ #define X86_FEATURE_BTS ( 3*32+13)/* "bts" Branch Trace Store */ #define X86_FEATURE_SYSCALL32 ( 3*32+14)/* syscall in IA32 userspace */ -#define X86_FEATURE_SYSENTER32 ( 3*32+15)/* sysenter in IA32 userspace */ +#define X86_FEATURE_SYSFAST32 ( 3*32+15)/* sysenter/syscall in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16)/* "rep_good" REP microcode works well */ #define X86_FEATURE_AMD_LBR_V2 ( 3*32+17)/* "amd_lbr_v2" AMD Last Branch Record Extension Version 2 */ #define X86_FEATURE_CLEAR_CPU_BUF ( 3*32+18)/* Clear CPU buffers using VERW */
arch/x86/kernel/cpu/centaur.c +0 -3
@@ -102,9 +102,6 @@ static void early_init_centaur(struct cpuinfo_x86 *c) (c->x86 >= 7))set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC ); -#ifdef CONFIG_X86_64 -set_cpu_cap (c,X86_FEATURE_SYSENTER32 ); -#endif if (c->x86_power & (1 <<8 )) {set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC );set_cpu_cap (c,X86_FEATURE_NONSTOP_TSC );
arch/x86/kernel/cpu/common.c +8 -0
@@ -1068,6 +1068,9 @@ void get_cpu_cap(struct cpuinfo_x86 *c)init_scattered_cpuid_features (c);init_speculation_control (c); +if (IS_ENABLED (CONFIG_X86_64 ) ||cpu_has (c,X86_FEATURE_SEP )) +set_cpu_cap (c,X86_FEATURE_SYSFAST32 ); +/* * Clear/Set all flags overridden by options, after probe. * This needs to happen each time we re-probe, which may happen
@@ -1813,6 +1816,11 @@ static void __init early_identify_cpu(struct cpuinfo_x86 *c)* that it can't be enabled in 32-bit mode. */ setup_clear_cpu_cap (X86_FEATURE_PCID ); + +/* +* Never use SYSCALL on a 32-bit kernel +*/ +setup_clear_cpu_cap (X86_FEATURE_SYSCALL32 );#endif /*
arch/x86/kernel/cpu/intel.c +1 -3
@@ -236,9 +236,7 @@ static void early_init_intel(struct cpuinfo_x86 *c)clear_cpu_cap (c,X86_FEATURE_PSE ); } -#ifdef CONFIG_X86_64 -set_cpu_cap (c,X86_FEATURE_SYSENTER32 ); -#else +#ifndef CONFIG_X86_64 /* Netburst reports 64 bytes clflush size, but does IO in 128 bytes */ if (c->x86 ==15 && c->x86_cache_alignment ==64 ) c->x86_cache_alignment =128 ;
arch/x86/kernel/cpu/zhaoxin.c +1 -3
@@ -59,9 +59,7 @@ static void early_init_zhaoxin(struct cpuinfo_x86 *c) {if (c->x86 >=0x6 )set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC ); -#ifdef CONFIG_X86_64 -set_cpu_cap (c,X86_FEATURE_SYSENTER32 ); -#endif +if (c->x86_power & (1 <<8 )) {set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC );set_cpu_cap (c,X86_FEATURE_NONSTOP_TSC );
arch/x86/kernel/fred.c +1 -1
@@ -68,7 +68,7 @@ void cpu_init_fred_exceptions(void)idt_invalidate ();/* Use int $0x80 for 32-bit system calls in FRED mode */ -setup_clear_cpu_cap (X86_FEATURE_SYSENTER32 ); +setup_clear_cpu_cap (X86_FEATURE_SYSFAST32 );setup_clear_cpu_cap (X86_FEATURE_SYSCALL32 ); }
arch/x86/xen/setup.c +18 -10
@@ -990,13 +990,6 @@ static int register_callback(unsigned type, const void *func)return HYPERVISOR_callback_op (CALLBACKOP_register, &callback); } -void xen_enable_sysenter (void ) -{ -if (cpu_feature_enabled (X86_FEATURE_SYSENTER32 ) && -register_callback (CALLBACKTYPE_sysenter, xen_entry_SYSENTER_compat)) -setup_clear_cpu_cap (X86_FEATURE_SYSENTER32 ); -} -void xen_enable_syscall (void ) {int ret;
@@ -1008,11 +1001,27 @@ void xen_enable_syscall(void)mechanism for syscalls. */ } -if (cpu_feature_enabled (X86_FEATURE_SYSCALL32 ) && -register_callback (CALLBACKTYPE_syscall32, xen_entry_SYSCALL_compat)) +if (!cpu_feature_enabled (X86_FEATURE_SYSFAST32 )) +return ; + +if (cpu_feature_enabled (X86_FEATURE_SYSCALL32 )) { +/* Use SYSCALL32 */ + ret =register_callback (CALLBACKTYPE_syscall32, + xen_entry_SYSCALL_compat); + + }else { +/* Use SYSENTER32 */ + ret =register_callback (CALLBACKTYPE_sysenter, + xen_entry_SYSENTER_compat); + } + +if (ret) {setup_clear_cpu_cap (X86_FEATURE_SYSCALL32 ); +setup_clear_cpu_cap (X86_FEATURE_SYSFAST32 ); + } } +static void __init xen_pvmmu_arch_setup (void ) {HYPERVISOR_vm_assist (VMASST_CMD_enable, VMASST_TYPE_writable_pagetables);
@@ -1022,7 +1031,6 @@ static void __init xen_pvmmu_arch_setup(void) register_callback(CALLBACKTYPE_failsafe, xen_failsafe_callback))BUG (); -xen_enable_sysenter ();xen_enable_syscall (); }
arch/x86/xen/smp_pv.c +2 -3
@@ -65,10 +65,9 @@ static void cpu_bringup(void)touch_softlockup_watchdog ();/* PVH runs in ring 0 and allows us to do native syscalls. Yay! */ -if (!xen_feature (XENFEAT_supervisor_mode_kernel)) { -xen_enable_sysenter (); +if (!xen_feature (XENFEAT_supervisor_mode_kernel))xen_enable_syscall (); - } + cpu =smp_processor_id ();identify_secondary_cpu (cpu);set_cpu_sibling_map (cpu);
arch/x86/xen/xen-ops.h +0 -1
@@ -60,7 +60,6 @@ phys_addr_t __init xen_find_free_area(phys_addr_t size);char *__init xen_memory_setup (void );void __init xen_arch_setup (void );void xen_banner (void ); -void xen_enable_sysenter (void );void xen_enable_syscall (void );void xen_vcpu_restore (void );
H. Peter Anvin
maintainerPATCH v4 10/10 x86/entry/vdso32: when using int $0x80, use it directly
In reply to H. Peter AnvinWhen neither sysenter32 nor syscall32 is available (on either FRED-capable 64-bit hardware or old 32-bit hardware), there is no reason to do a bunch of stack shuffling in __kernel_vsyscall. Unfortunately, just overwriting the initial "push" instructions will mess up the CFI annotations, so suffer the 3-byte NOP if not applicable.
Similarly, inline the int $0x80 when doing inline system calls in the vdso instead of calling __kernel_vsyscall.
--- arch/x86/entry/vdso/vdso32/system_call.S | 18 ++++++++++++++---- arch/x86/include/asm/vdso/sys_call.h | 4 +++- 2 files changed, 17 insertions(+), 5 deletions(-)
arch/x86/entry/vdso/vdso32/system_call.S +14 -4
@@ -14,6 +14,18 @@ ALIGN __kernel_vsyscall: CFI_STARTPROC + + /* + * If using int $0x80, there is no reason to muck about with the + * stack here. Unfortunately just overwriting the push instructions + * would mess up the CFI annotations, but it is only a 3-byte + * NOP in that case. This could be avoided by patching the + * vdso symbol table (not the code) and entry point, but that + * would a fair bit of tooling work or by simply compiling + * two different vDSO images, but that doesn't seem worth it. + */ + ALTERNATIVE "int $0x80; ret", "", X86_FEATURE_SYSFAST32 + /* * Reshuffle regs so that all of any of the entry instructions * will preserve enough state.
@@ -52,11 +64,9 @@ __kernel_vsyscall: #define SYSENTER_SEQUENCE "movl %esp, %ebp; sysenter" #define SYSCALL_SEQUENCE "movl %ecx, %ebp; syscall" - /* If SYSENTER (Intel) or SYSCALL32 (AMD) is available, use it. */ - ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSFAST32, \ - SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 + ALTERNATIVE SYSENTER_SEQUENCE, SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 - /* Enter using int $0x80 */ + /* Re-enter using int $0x80 */ int $0x80 SYM_INNER_LABEL(int80_landing_pad, SYM_L_GLOBAL)
arch/x86/include/asm/vdso/sys_call.h +3 -1
@@ -20,7 +20,9 @@# define __sys_reg4 "r10"# define __sys_reg5 "r8"#else -# define __sys_instr "call __kernel_vsyscall" +# define __sys_instr ALTERNATIVE("ds;ds;ds;int $0x80", \ + "call __kernel_vsyscall", \ + X86_FEATURE_SYSFAST32)# define __sys_clobber "memory"# define __sys_nr (x,y) __NR_ ## x ## y# define __sys_reg1 "ebx"
Signature · 1 line
2.52.0
[tip: x86/entry] x86/entry/vdso32: When using int $0x80, use it directly
The following commit has been merged into the x86/entry branch of tip:
The following commit has been merged into the x86/entry branch of tip:
Commit-ID: 36d83c249e0395a915144eceeb528ddc19b1fbe6 Gitweb: https://git.kernel.org/tip/36d83c249e0395a915144eceeb528ddc19b1fbe6 Author: H. Peter Anvin <[email protected]> AuthorDate: Tue, 16 Dec 2025 13:26:04 -08:00 Committer: Dave Hansen <[email protected]> CommitterDate: Tue, 13 Jan 2026 16:37:58 -08:00
x86/entry/vdso32: When using int $0x80, use it directly
When neither sysenter32 nor syscall32 is available (on either FRED-capable 64-bit hardware or old 32-bit hardware), there is no reason to do a bunch of stack shuffling in __kernel_vsyscall. Unfortunately, just overwriting the initial "push" instructions will mess up the CFI annotations, so suffer the 3-byte NOP if not applicable.
Similarly, inline the int $0x80 when doing inline system calls in the vdso instead of calling __kernel_vsyscall.
--- arch/x86/entry/vdso/vdso32/system_call.S | 18 ++++++++++++++---- arch/x86/include/asm/vdso/sys_call.h | 4 +++- 2 files changed, 17 insertions(+), 5 deletions(-)
arch/x86/entry/vdso/vdso32/system_call.S +14 -4
@@ -14,6 +14,18 @@ ALIGN __kernel_vsyscall: CFI_STARTPROC + + /* + * If using int $0x80, there is no reason to muck about with the + * stack here. Unfortunately just overwriting the push instructions + * would mess up the CFI annotations, but it is only a 3-byte + * NOP in that case. This could be avoided by patching the + * vdso symbol table (not the code) and entry point, but that + * would a fair bit of tooling work or by simply compiling + * two different vDSO images, but that doesn't seem worth it. + */ + ALTERNATIVE "int $0x80; ret", "", X86_FEATURE_SYSFAST32 + /* * Reshuffle regs so that all of any of the entry instructions * will preserve enough state.
@@ -52,11 +64,9 @@ __kernel_vsyscall: #define SYSENTER_SEQUENCE "movl %esp, %ebp; sysenter" #define SYSCALL_SEQUENCE "movl %ecx, %ebp; syscall" - /* If SYSENTER (Intel) or SYSCALL32 (AMD) is available, use it. */ - ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSFAST32, \ - SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 + ALTERNATIVE SYSENTER_SEQUENCE, SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 - /* Enter using int $0x80 */ + /* Re-enter using int $0x80 */ int $0x80 SYM_INNER_LABEL(int80_landing_pad, SYM_L_GLOBAL)
arch/x86/include/asm/vdso/sys_call.h +3 -1
@@ -20,7 +20,9 @@# define __sys_reg4 "r10"# define __sys_reg5 "r8"#else -# define __sys_instr "call __kernel_vsyscall" +# define __sys_instr ALTERNATIVE("ds;ds;ds;int $0x80", \ + "call __kernel_vsyscall", \ + X86_FEATURE_SYSFAST32)# define __sys_clobber "memory"# define __sys_nr (x,y) __NR_ ## x ## y# define __sys_reg1 "ebx"
H. Peter Anvin
maintainercover x86/entry/vdso: clean up the vdso build, vdso updates
In reply to H. Peter AnvinThis is intended as a ping, since I think the v4 got swallowed by the holidays. v4.1 IS BASICALLY A REBASE AND RESEND OF v4; THE ONLY CODE CHANGE IS A SINGLE SPACE CHARACTER.
This patchset cleans up the vdso build by building the 32- and 64-bit vdsos in separate directories, moving the vdso2c tool to arch/x86/tools, and by merging common code and especially Makefile rules between the 32- and 64-bit vdsos to the greatest extent possible.
Be more strict about sanitizing and standardizing the vdso build options.
Disable CFI for the vdso until the kernel adds user space IBT support.
Modernize the DWARF generation in vdso32/sigreturn.S.
Add macros to cleanly call system calls from vdso source code.
Add the GNU_PROPERTY and GNU_STACK PHDRs to the vdso.
When using int $0x80 (old 32-bit hardware or FRED-capable 64-bit hardware) skip the stack stuff in the 32-bit kernel entry code and call int $0x80 directly when used from C code.
Changes from v4 to v4.1:
- Fix a single bogus whitespace character change in patch 7.
- Fix the spelling of Uros Bizjak's name in the comment to patch 8.
- Rebased onto v6.19-rc4.
Changes from v3 to v4:
- Improved description of patch 01/10.
- Split out the move of vdso2c to tools into a separate patch.
- Remove unrelated formatting changes from vdso-layout.lds.S.
- Fix *-x32.o being missing from "targets".
- Rebased onto v6.19-rc1.
Changes from v2 to v3:
In arch/x86/include/asm/vdso/sys_call.h:
- Remove stray comment from prototyping (Uros Bizjak)
- Remove VDSO_SYSCALL6() since it would require special handling on 32 bits with frame pointers and is currently unused. (Uros Bizjak)
- Indent nested preprocessor directives.
Changes from v1 to v2:
Too many to count - much of the patchset has been reworked
Patches:
x86/entry/vdso: rename vdso_image_* to vdso*_image x86/entry/vdso: move vdso2c to arch/x86/tools
x86/entry/vdso: refactor the vdso build
x86/entry/vdso32: don't rely on int80_landing_pad for adjusting ip x86/entry/vdso32: remove SYSCALL_ENTER_KERNEL macro in sigreturn.S x86/entry/vdso32: remove open-coded DWARF in sigreturn.S x86/entry/vdso: include GNU_PROPERTY and GNU_STACK PHDRs x86/vdso: abstract out vdso system call internals x86/cpufeature: replace X86_FEATURE_SYSENTER32 with X86_FEATURE_SYSFAST32 x86/entry/vdso32: when using int $0x80, use it directly
---
arch/x86/Kconfig.cpufeatures | 8 +
arch/x86/Makefile | 2 +-
arch/x86/entry/syscall_32.c | 2 +-
arch/x86/entry/vdso/.gitignore | 11 +-
arch/x86/entry/vdso/Makefile | 162 +--------------------
arch/x86/entry/vdso/common/Makefile.include | 89 +++++++++++
arch/x86/entry/vdso/{vdso-note.S => common/note.S} | 5 +-
arch/x86/entry/vdso/{ => common}/vclock_gettime.c | 0
arch/x86/entry/vdso/{ => common}/vdso-layout.lds.S | 38 +++--
arch/x86/entry/vdso/{ => common}/vgetcpu.c | 0
arch/x86/entry/vdso/vdso32/Makefile | 24 +++
arch/x86/entry/vdso/vdso32/note.S | 19 +--
arch/x86/entry/vdso/vdso32/sigreturn.S | 152 +++++--------------
arch/x86/entry/vdso/vdso32/system_call.S | 22 ++-
arch/x86/entry/vdso/vdso32/vclock_gettime.c | 5 +-
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 2 +-
arch/x86/entry/vdso/vdso32/vgetcpu.c | 4 +-
arch/x86/entry/vdso/vdso64/Makefile | 46 ++++++
arch/x86/entry/vdso/vdso64/note.S | 1 +
arch/x86/entry/vdso/vdso64/vclock_gettime.c | 1 +
.../entry/vdso/{vdso.lds.S => vdso64/vdso64.lds.S} | 2 +-
arch/x86/entry/vdso/{ => vdso64}/vdsox32.lds.S | 2 +-
arch/x86/entry/vdso/vdso64/vgetcpu.c | 1 +
.../entry/vdso/{ => vdso64}/vgetrandom-chacha.S | 0
arch/x86/entry/vdso/{ => vdso64}/vgetrandom.c | 2 +-
arch/x86/entry/vdso/{ => vdso64}/vsgx.S | 0
arch/x86/entry/vdso/vma.c | 24 ++-
arch/x86/include/asm/cpufeatures.h | 2 +-
arch/x86/include/asm/dwarf2.h | 1 +
arch/x86/include/asm/elf.h | 2 +-
arch/x86/include/asm/vdso.h | 6 +-
arch/x86/include/asm/vdso/gettimeofday.h | 108 +-------------
arch/x86/include/asm/vdso/sys_call.h | 105 +++++++++++++
arch/x86/kernel/asm-offsets.c | 6 +
arch/x86/kernel/cpu/centaur.c | 3 -
arch/x86/kernel/cpu/common.c | 8 +
arch/x86/kernel/cpu/intel.c | 4 +-
arch/x86/kernel/cpu/zhaoxin.c | 4 +-
arch/x86/kernel/fred.c | 2 +-
arch/x86/kernel/process_64.c | 6 +-
arch/x86/kernel/signal_32.c | 4 +-
arch/x86/tools/Makefile | 15 +-
arch/x86/{entry/vdso => tools}/vdso2c.c | 0
arch/x86/{entry/vdso => tools}/vdso2c.h | 0
arch/x86/xen/setup.c | 28 ++--
arch/x86/xen/smp_pv.c | 5 +-
arch/x86/xen/xen-ops.h | 1 -
47 files changed, 444 insertions(+), 490 deletions(-)H. Peter Anvin
maintainerPATCH v4 1/10 x86/entry/vdso: rename vdso_image_* to vdso*_image
The vdso .so files are named vdso*.so. These structures are binary images and descriptions of these files, so it is more consistent for them to have a naming that more directly mirrors the filenames.
It is also very slightly more compact (by one character...) and simplifies the Makefile just a little bit.
--- arch/x86/entry/syscall_32.c | 2 +- arch/x86/entry/vdso/.gitignore | 11 ++++------- arch/x86/entry/vdso/Makefile | 8 ++++---- arch/x86/entry/vdso/vma.c | 10 +++++----- arch/x86/include/asm/elf.h | 2 +- arch/x86/include/asm/vdso.h | 6 +++--- arch/x86/kernel/process_64.c | 6 +++--- arch/x86/kernel/signal_32.c | 4 ++-- 8 files changed, 23 insertions(+), 26 deletions(-)
arch/x86/entry/syscall_32.c +1 -1
@@ -319,7 +319,7 @@ __visible noinstr bool do_fast_syscall_32(struct pt_regs *regs)* convention. Adjust regs so it looks like we entered using int80. */ unsigned long landing_pad = (unsigned long )current->mm->context.vdso + - vdso_image_32.sym_int80_landing_pad; + vdso32_image.sym_int80_landing_pad;/* * SYSENTER loses EIP, and even SYSCALL32 needs us to skip forward
arch/x86/entry/vdso/.gitignore +4 -7
@@ -1,8 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -vdso.lds -vdsox32.lds -vdso32-syscall-syms.lds -vdso32-sysenter-syms.lds -vdso32-int80-syms.lds -vdso-image-*.c -vdso2c +*.lds +*.so +*.so.dbg +vdso*-image.c
arch/x86/entry/vdso/Makefile +4 -4
@@ -16,9 +16,9 @@ vobjs-$(CONFIG_X86_SGX) += vsgx.o obj-y += vma.o extable.o # vDSO images to build: -obj-$(CONFIG_X86_64) += vdso-image-64.o -obj-$(CONFIG_X86_X32_ABI) += vdso-image-x32.o -obj-$(CONFIG_COMPAT_32) += vdso-image-32.o vdso32-setup.o +obj-$(CONFIG_X86_64) += vdso64-image.o +obj-$(CONFIG_X86_X32_ABI) += vdsox32-image.o +obj-$(CONFIG_COMPAT_32) += vdso32-image.o vdso32-setup.o vobjs := $(addprefix $(obj)/, $(vobjs-y)) vobjs32 := $(addprefix $(obj)/, $(vobjs32-y))
@@ -44,7 +44,7 @@ hostprogs += vdso2c quiet_cmd_vdso2c = VDSO2C $@ cmd_vdso2c = $(obj)/vdso2c $< $(<:%.dbg=%) $@ -$(obj)/vdso-image-%.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE +$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE $(call if_changed,vdso2c) #
arch/x86/entry/vdso/vma.c +5 -5
@@ -65,7 +65,7 @@ static vm_fault_t vdso_fault(const struct vm_special_mapping *sm,static void vdso_fix_landing (const struct vdso_image *image,struct vm_area_struct *new_vma) { -if (in_ia32_syscall () && image == &vdso_image_32) { +if (in_ia32_syscall () && image == &vdso32_image) {struct pt_regs *regs =current_pt_regs ();unsigned long vdso_land = image->sym_int80_landing_pad; unsigned long old_land_addr = vdso_land +
@@ -230,7 +230,7 @@ static int load_vdso32(void)if (vdso32_enabled !=1 )/* Other values all mean "disabled" */ return 0 ; -return map_vdso (&vdso_image_32,0 ); +return map_vdso (&vdso32_image,0 ); }int arch_setup_additional_pages (struct linux_binprm *bprm,int uses_interp)
@@ -239,7 +239,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)if (!vdso64_enabled)return 0 ; -return map_vdso (&vdso_image_64,0 ); +return map_vdso (&vdso64_image,0 ); }return load_vdso32 ();
@@ -252,7 +252,7 @@ int compat_arch_setup_additional_pages(struct linux_binprm *bprm,if (IS_ENABLED (CONFIG_X86_X32_ABI ) && x32) {if (!vdso64_enabled)return 0 ; -return map_vdso (&vdso_image_x32,0 ); +return map_vdso (&vdsox32_image,0 ); }if (IS_ENABLED (CONFIG_IA32_EMULATION ))
@@ -267,7 +267,7 @@ bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs)const struct vdso_image *image = current->mm->context.vdso_image;unsigned long vdso = (unsigned long ) current->mm->context.vdso; -if (in_ia32_syscall () && image == &vdso_image_32) { +if (in_ia32_syscall () && image == &vdso32_image) {if (regs->ip == vdso + image->sym_vdso32_sigreturn_landing_pad || regs->ip == vdso + image->sym_vdso32_rt_sigreturn_landing_pad)return true;
arch/x86/include/asm/elf.h +1 -1
@@ -361,7 +361,7 @@ else if (IS_ENABLED(CONFIG_IA32_EMULATION)) \#define VDSO_ENTRY \ ((unsigned long)current->mm->context.vdso + \ - vdso_image_32.sym___kernel_vsyscall) + vdso32_image.sym___kernel_vsyscall)struct linux_binprm ;
arch/x86/include/asm/vdso.h +3 -3
@@ -27,9 +27,9 @@ struct vdso_image {long sym_vdso32_rt_sigreturn_landing_pad; }; -extern const struct vdso_image vdso_image_64; -extern const struct vdso_image vdso_image_x32; -extern const struct vdso_image vdso_image_32; +extern const struct vdso_image vdso64_image; +extern const struct vdso_image vdsox32_image; +extern const struct vdso_image vdso32_image;extern int __init init_vdso_image (const struct vdso_image *image);
arch/x86/kernel/process_64.c +3 -3
@@ -941,14 +941,14 @@ long do_arch_prctl_64(struct task_struct *task, int option, unsigned long arg2)#ifdef CONFIG_CHECKPOINT_RESTORE # ifdef CONFIG_X86_X32_ABI case ARCH_MAP_VDSO_X32 : -return prctl_map_vdso (&vdso_image_x32, arg2); +return prctl_map_vdso (&vdsox32_image, arg2);# endif # ifdef CONFIG_IA32_EMULATION case ARCH_MAP_VDSO_32 : -return prctl_map_vdso (&vdso_image_32, arg2); +return prctl_map_vdso (&vdso32_image, arg2);# endif case ARCH_MAP_VDSO_64 : -return prctl_map_vdso (&vdso_image_64, arg2); +return prctl_map_vdso (&vdso64_image, arg2);#endif #ifdef CONFIG_ADDRESS_MASKING case ARCH_GET_UNTAG_MASK :
arch/x86/kernel/signal_32.c +2 -2
@@ -282,7 +282,7 @@ int ia32_setup_frame(struct ksignal *ksig, struct pt_regs *regs)/* Return stub is in 32bit vsyscall page */ if (current->mm->context.vdso) restorer = current->mm->context.vdso + - vdso_image_32.sym___kernel_sigreturn; + vdso32_image.sym___kernel_sigreturn;else restorer = &frame->retcode; }
@@ -368,7 +368,7 @@ int ia32_setup_rt_frame(struct ksignal *ksig, struct pt_regs *regs) restorer = ksig->ka.sa.sa_restorer; else restorer = current->mm->context.vdso + - vdso_image_32.sym___kernel_rt_sigreturn; + vdso32_image.sym___kernel_rt_sigreturn;unsafe_put_user (ptr_to_compat (restorer), &frame->pretcode, Efault);/*
Signature · 1 line
2.52.0
H. Peter Anvin
maintainerPATCH v4 2/10 x86/entry/vdso: move vdso2c to arch/x86/tools
In reply to H. Peter AnvinIt is generally better to build tools in arch/x86/tools to keep host cflags proliferation down, and to reduce makefile sequencing issues. Move the vdso build tool vdso2c into arch/x86/tools in preparation for refactoring the vdso makefiles.
---
arch/x86/Makefile | 2 +-
arch/x86/entry/vdso/Makefile | 7 +++----
arch/x86/tools/Makefile | 15 ++++++++++-----
arch/x86/{entry/vdso => tools}/vdso2c.c | 0
arch/x86/{entry/vdso => tools}/vdso2c.h | 0
5 files changed, 14 insertions(+), 10 deletions(-)
rename arch/x86/{entry/vdso => tools}/vdso2c.c (100%)
rename arch/x86/{entry/vdso => tools}/vdso2c.h (100%)arch/x86/Makefile +1 -1
@@ -252,7 +252,7 @@ endif archscripts: scripts_basic - $(Q)$(MAKE) $(build)=arch/x86/tools relocs + $(Q)$(MAKE) $(build)=arch/x86/tools relocs vdso2c ### # Syscall table generation
arch/x86/entry/vdso/Makefile +3 -4
@@ -38,13 +38,12 @@ VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 \ $(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE $(call if_changed,vdso_and_check) -HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi -I$(srctree)/arch/$(SUBARCH)/include/uapi -hostprogs += vdso2c +VDSO2C = $(objtree)/arch/x86/tools/vdso2c quiet_cmd_vdso2c = VDSO2C $@ - cmd_vdso2c = $(obj)/vdso2c $< $(<:%.dbg=%) $@ + cmd_vdso2c = $(VDSO2C) $< $(<:%.dbg=%) $@ -$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE +$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(VDSO2C) FORCE $(call if_changed,vdso2c) #
arch/x86/tools/Makefile +10 -5
@@ -38,9 +38,14 @@ $(obj)/insn_decoder_test.o: $(srctree)/tools/arch/x86/lib/insn.c $(srctree)/tool $(obj)/insn_sanity.o: $(srctree)/tools/arch/x86/lib/insn.c $(srctree)/tools/arch/x86/lib/inat.c $(srctree)/tools/arch/x86/include/asm/inat_types.h $(srctree)/tools/arch/x86/include/asm/inat.h $(srctree)/tools/arch/x86/include/asm/insn.h $(objtree)/arch/x86/lib/inat-tables.c -HOST_EXTRACFLAGS += -I$(srctree)/tools/include -hostprogs += relocs -relocs-objs := relocs_32.o relocs_64.o relocs_common.o -PHONY += relocs -relocs: $(obj)/relocs +HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi \ + -I$(srctree)/arch/$(SUBARCH)/include/uapi + +hostprogs += relocs vdso2c +relocs-objs := relocs_32.o relocs_64.o relocs_common.o + +always-y := $(hostprogs) + +PHONY += $(hostprogs) +$(hostprogs): %: $(obj)/% @:
arch/x86/tools/vdso2c.c renamed from arch/x86/entry/vdso/vdso2c.c +0 -0
arch/x86/tools/vdso2c.h renamed from arch/x86/entry/vdso/vdso2c.h +0 -0
Signature · 1 line
2.52.0
H. Peter Anvin
maintainerPATCH v4 3/10 x86/entry/vdso: refactor the vdso build
In reply to H. Peter Anvin- Separate out the vdso sources into common, vdso32, and vdso64 directories.
- Build the 32- and 64-bit vdsos in their respective subdirectories; this greatly simplifies the build flags handling.
- Unify the mangling of Makefile flags between the 32- and 64-bit vdso code as much as possible; all common rules are put in arch/x86/entry/vdso/common/Makefile.include. The remaining is very simple for 32 bits; the 64-bit one is only slightly more complicated because it contains the x32 generation rule.
- Define __DISABLE_EXPORTS when building the vdso. This need seems to have been masked by different ordering compile flags before.
- Change CONFIG_X86_64 to BUILD_VDSO32_64 in vdso32/system_call.S, to make it compatible with including fake_32bit_build.h.
- The -fcf-protection= option was "leaking" from the kernel build, for reasons that was not clear to me. Futhermore, several distributions ship with it set to a default value other than "-fcf-protection=none". Make it match the configuration options for *user space*.
Note that this patch may seem large, but the vast majority of it is simply code movement.
---
arch/x86/entry/vdso/Makefile | 161 +-----------------
arch/x86/entry/vdso/common/Makefile.include | 89 ++++++++++
.../entry/vdso/{vdso-note.S => common/note.S} | 5 +-
.../entry/vdso/{ => common}/vclock_gettime.c | 0
.../entry/vdso/{ => common}/vdso-layout.lds.S | 0
arch/x86/entry/vdso/{ => common}/vgetcpu.c | 0
arch/x86/entry/vdso/vdso32/Makefile | 24 +++
arch/x86/entry/vdso/vdso32/note.S | 19 +--
arch/x86/entry/vdso/vdso32/system_call.S | 2 +-
arch/x86/entry/vdso/vdso32/vclock_gettime.c | 5 +-
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 2 +-
arch/x86/entry/vdso/vdso32/vgetcpu.c | 4 +-
arch/x86/entry/vdso/vdso64/Makefile | 46 +++++
arch/x86/entry/vdso/vdso64/note.S | 1 +
arch/x86/entry/vdso/vdso64/vclock_gettime.c | 1 +
.../vdso/{vdso.lds.S => vdso64/vdso64.lds.S} | 2 +-
.../x86/entry/vdso/{ => vdso64}/vdsox32.lds.S | 2 +-
arch/x86/entry/vdso/vdso64/vgetcpu.c | 1 +
.../vdso/{ => vdso64}/vgetrandom-chacha.S | 0
arch/x86/entry/vdso/{ => vdso64}/vgetrandom.c | 2 +-
arch/x86/entry/vdso/{ => vdso64}/vsgx.S | 0
21 files changed, 180 insertions(+), 186 deletions(-)
create mode 100644 arch/x86/entry/vdso/common/Makefile.include
rename arch/x86/entry/vdso/{vdso-note.S => common/note.S} (62%)
rename arch/x86/entry/vdso/{ => common}/vclock_gettime.c (100%)
rename arch/x86/entry/vdso/{ => common}/vdso-layout.lds.S (100%)
rename arch/x86/entry/vdso/{ => common}/vgetcpu.c (100%)
create mode 100644 arch/x86/entry/vdso/vdso32/Makefile
create mode 100644 arch/x86/entry/vdso/vdso64/Makefile
create mode 100644 arch/x86/entry/vdso/vdso64/note.S
create mode 100644 arch/x86/entry/vdso/vdso64/vclock_gettime.c
rename arch/x86/entry/vdso/{vdso.lds.S => vdso64/vdso64.lds.S} (94%)
rename arch/x86/entry/vdso/{ => vdso64}/vdsox32.lds.S (92%)
create mode 100644 arch/x86/entry/vdso/vdso64/vgetcpu.c
rename arch/x86/entry/vdso/{ => vdso64}/vgetrandom-chacha.S (100%)
rename arch/x86/entry/vdso/{ => vdso64}/vgetrandom.c (91%)
rename arch/x86/entry/vdso/{ => vdso64}/vsgx.S (100%)arch/x86/entry/vdso/Makefile +6 -155
@@ -3,159 +3,10 @@ # Building vDSO images for x86. # -# Include the generic Makefile to check the built vDSO: -include $(srctree)/lib/vdso/Makefile.include +# Regular kernel objects +obj-y := vma.o extable.o +obj-$(CONFIG_COMPAT_32) += vdso32-setup.o -# Files to link into the vDSO: -vobjs-y := vdso-note.o vclock_gettime.o vgetcpu.o vgetrandom.o vgetrandom-chacha.o -vobjs32-y := vdso32/note.o vdso32/system_call.o vdso32/sigreturn.o -vobjs32-y += vdso32/vclock_gettime.o vdso32/vgetcpu.o -vobjs-$(CONFIG_X86_SGX) += vsgx.o - -# Files to link into the kernel: -obj-y += vma.o extable.o - -# vDSO images to build: -obj-$(CONFIG_X86_64) += vdso64-image.o -obj-$(CONFIG_X86_X32_ABI) += vdsox32-image.o -obj-$(CONFIG_COMPAT_32) += vdso32-image.o vdso32-setup.o - -vobjs := $(addprefix $(obj)/, $(vobjs-y)) -vobjs32 := $(addprefix $(obj)/, $(vobjs32-y)) - -$(obj)/vdso.o: $(obj)/vdso.so - -targets += vdso.lds $(vobjs-y) -targets += vdso32/vdso32.lds $(vobjs32-y) - -targets += $(foreach x, 64 x32 32, vdso-image-$(x).c vdso$(x).so vdso$(x).so.dbg) - -CPPFLAGS_vdso.lds += -P -C - -VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 \ - -z max-page-size=4096 - -$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE - $(call if_changed,vdso_and_check) - -VDSO2C = $(objtree)/arch/x86/tools/vdso2c - -quiet_cmd_vdso2c = VDSO2C $@ - cmd_vdso2c = $(VDSO2C) $< $(<:%.dbg=%) $@ - -$(obj)/vdso%-image.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(VDSO2C) FORCE - $(call if_changed,vdso2c) - -# -# Don't omit frame pointers for ease of userspace debugging, but do -# optimize sibling calls. -# -CFL := $(PROFILING) -mcmodel=small -fPIC -O2 -fasynchronous-unwind-tables -m64 \ - $(filter -g%,$(KBUILD_CFLAGS)) -fno-stack-protector \ - -fno-omit-frame-pointer -foptimize-sibling-calls \ - -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO - -ifdef CONFIG_MITIGATION_RETPOLINE -ifneq ($(RETPOLINE_VDSO_CFLAGS),) - CFL += $(RETPOLINE_VDSO_CFLAGS) -endif -endif - -$(vobjs): KBUILD_CFLAGS := $(filter-out $(PADDING_CFLAGS) $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) $(RANDSTRUCT_CFLAGS) $(KSTACK_ERASE_CFLAGS) $(GCC_PLUGINS_CFLAGS) $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS)) $(CFL) -$(vobjs): KBUILD_AFLAGS += -DBUILD_VDSO - -# -# vDSO code runs in userspace and -pg doesn't help with profiling anyway. -# -CFLAGS_REMOVE_vclock_gettime.o = -pg -CFLAGS_REMOVE_vdso32/vclock_gettime.o = -pg -CFLAGS_REMOVE_vgetcpu.o = -pg -CFLAGS_REMOVE_vdso32/vgetcpu.o = -pg -CFLAGS_REMOVE_vsgx.o = -pg -CFLAGS_REMOVE_vgetrandom.o = -pg - -# -# X32 processes use x32 vDSO to access 64bit kernel data. -# -# Build x32 vDSO image: -# 1. Compile x32 vDSO as 64bit. -# 2. Convert object files to x32. -# 3. Build x32 VDSO image with x32 objects, which contains 64bit codes -# so that it can reach 64bit address space with 64bit pointers. -# - -CPPFLAGS_vdsox32.lds = $(CPPFLAGS_vdso.lds) -VDSO_LDFLAGS_vdsox32.lds = -m elf32_x86_64 -soname linux-vdso.so.1 \ - -z max-page-size=4096 - -# x32-rebranded versions -vobjx32s-y := $(vobjs-y:.o=-x32.o) - -# same thing, but in the output directory -vobjx32s := $(addprefix $(obj)/, $(vobjx32s-y)) - -# Convert 64bit object file to x32 for x32 vDSO. -quiet_cmd_x32 = X32 $@ - cmd_x32 = $(OBJCOPY) -O elf32-x86-64 $< $@ - -$(obj)/%-x32.o: $(obj)/%.o FORCE - $(call if_changed,x32) - -targets += vdsox32.lds $(vobjx32s-y) - -$(obj)/%.so: OBJCOPYFLAGS := -S --remove-section __ex_table -$(obj)/%.so: $(obj)/%.so.dbg FORCE - $(call if_changed,objcopy) - -$(obj)/vdsox32.so.dbg: $(obj)/vdsox32.lds $(vobjx32s) FORCE - $(call if_changed,vdso_and_check) - -CPPFLAGS_vdso32/vdso32.lds = $(CPPFLAGS_vdso.lds) -VDSO_LDFLAGS_vdso32.lds = -m elf_i386 -soname linux-gate.so.1 - -KBUILD_AFLAGS_32 := $(filter-out -m64,$(KBUILD_AFLAGS)) -DBUILD_VDSO -$(obj)/vdso32.so.dbg: KBUILD_AFLAGS = $(KBUILD_AFLAGS_32) -$(obj)/vdso32.so.dbg: asflags-$(CONFIG_X86_64) += -m32 - -KBUILD_CFLAGS_32 := $(filter-out -m64,$(KBUILD_CFLAGS)) -KBUILD_CFLAGS_32 := $(filter-out -mcmodel=kernel,$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out -fno-pic,$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out -mfentry,$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(RANDSTRUCT_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(KSTACK_ERASE_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(GCC_PLUGINS_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(CC_FLAGS_CFI),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 := $(filter-out $(PADDING_CFLAGS),$(KBUILD_CFLAGS_32)) -KBUILD_CFLAGS_32 += -m32 -msoft-float -mregparm=0 -fpic -KBUILD_CFLAGS_32 += -fno-stack-protector -KBUILD_CFLAGS_32 += $(call cc-option, -foptimize-sibling-calls) -KBUILD_CFLAGS_32 += -fno-omit-frame-pointer -KBUILD_CFLAGS_32 += -DDISABLE_BRANCH_PROFILING -KBUILD_CFLAGS_32 += -DBUILD_VDSO - -ifdef CONFIG_MITIGATION_RETPOLINE -ifneq ($(RETPOLINE_VDSO_CFLAGS),) - KBUILD_CFLAGS_32 += $(RETPOLINE_VDSO_CFLAGS) -endif -endif - -$(obj)/vdso32.so.dbg: KBUILD_CFLAGS = $(KBUILD_CFLAGS_32) - -$(obj)/vdso32.so.dbg: $(obj)/vdso32/vdso32.lds $(vobjs32) FORCE - $(call if_changed,vdso_and_check) - -# -# The DSO images are built using a special linker script. -# -quiet_cmd_vdso = VDSO $@ - cmd_vdso = $(LD) -o $@ \ - $(VDSO_LDFLAGS) $(VDSO_LDFLAGS_$(filter %.lds,$(^F))) \ - -T $(filter %.lds,$^) $(filter %.o,$^) - -VDSO_LDFLAGS = -shared --hash-style=both --build-id=sha1 --no-undefined \ - $(call ld-option, --eh-frame-hdr) -Bsymbolic -z noexecstack - -quiet_cmd_vdso_and_check = VDSO $@ - cmd_vdso_and_check = $(cmd_vdso); $(cmd_vdso_check) +# vDSO directories +obj-$(CONFIG_X86_64) += vdso64/ +obj-$(CONFIG_COMPAT_32) += vdso32/
arch/x86/entry/vdso/common/Makefile.include new file +89 -0
89 lines not on this page · raw message
arch/x86/entry/vdso/common/note.S renamed from arch/x86/entry/vdso/vdso-note.S +4 -1
@@ -1,13 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ /* * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text. * Here we can supply some information useful to userland. */ #include <linux/build-salt.h> -#include <linux/uts.h> #include <linux/version.h> #include <linux/elfnote.h> +/* Ideally this would use UTS_NAME, but using a quoted string here + doesn't work. Remember to change this when changing the + kernel's name. */ ELFNOTE_START(Linux, 0, "a") .long LINUX_VERSION_CODE ELFNOTE_END
arch/x86/entry/vdso/common/vclock_gettime.c renamed from arch/x86/entry/vdso/vclock_gettime.c +0 -0
arch/x86/entry/vdso/common/vdso-layout.lds.S renamed from arch/x86/entry/vdso/vdso-layout.lds.S +0 -0
arch/x86/entry/vdso/common/vgetcpu.c renamed from arch/x86/entry/vdso/vgetcpu.c +0 -0
arch/x86/entry/vdso/vdso32/Makefile new file +24 -0
@@ -0,0 +1,24 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# 32-bit vDSO images for x86. +# + +# The vDSOs built in this directory +vdsos-y := 32 + +# Files to link into the vDSO: +vobjs-y := note.o vclock_gettime.o vgetcpu.o +vobjs-y += system_call.o sigreturn.o + +# Compilation flags +flags-y := -DBUILD_VDSO32 -m32 -mregparm=0 +flags-$(CONFIG_X86_64) += -include $(src)/fake_32bit_build.h +flags-remove-y := -m64 + +# The location of this include matters! +include $(src)/../common/Makefile.include + +# Linker options for the vdso +VDSO_LDFLAGS_32 := -m elf_i386 -soname linux-gate.so.1 + +$(obj)/vdso32.so.dbg: $(vobjs)
arch/x86/entry/vdso/vdso32/note.S +1 -18
@@ -1,18 +1 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text. - * Here we can supply some information useful to userland. - */ - -#include <linux/build-salt.h> -#include <linux/version.h> -#include <linux/elfnote.h> - -/* Ideally this would use UTS_NAME, but using a quoted string here - doesn't work. Remember to change this when changing the - kernel's name. */ -ELFNOTE_START(Linux, 0, "a") - .long LINUX_VERSION_CODE -ELFNOTE_END - -BUILD_SALT +#include "common/note.S"
arch/x86/entry/vdso/vdso32/system_call.S +1 -1
@@ -52,7 +52,7 @@ __kernel_vsyscall: #define SYSENTER_SEQUENCE "movl %esp, %ebp; sysenter" #define SYSCALL_SEQUENCE "movl %ecx, %ebp; syscall" -#ifdef CONFIG_X86_64 +#ifdef BUILD_VDSO32_64 /* If SYSENTER (Intel) or SYSCALL32 (AMD) is available, use it. */ ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSENTER32, \ SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32
arch/x86/entry/vdso/vdso32/vclock_gettime.c +1 -4
@@ -1,4 +1 @@ -// SPDX-License-Identifier: GPL-2.0 -#define BUILD_VDSO32 -#include "fake_32bit_build.h" -#include "../vclock_gettime.c" +#include "common/vclock_gettime.c"
arch/x86/entry/vdso/vdso32/vdso32.lds.S +1 -1
@@ -11,7 +11,7 @@ #define BUILD_VDSO32 -#include "../vdso-layout.lds.S" +#include "common/vdso-layout.lds.S" /* The ELF entry point can be used to set the AT_SYSINFO value. */ ENTRY(__kernel_vsyscall);
arch/x86/entry/vdso/vdso32/vgetcpu.c +1 -3
@@ -1,3 +1 @@ -// SPDX-License-Identifier: GPL-2.0 -#include "fake_32bit_build.h" -#include "../vgetcpu.c" +#include "common/vgetcpu.c"
arch/x86/entry/vdso/vdso64/Makefile new file +46 -0
46 lines not on this page · raw message
arch/x86/entry/vdso/vdso64/note.S new file +1 -0
@@ -0,0 +1 @@ +#include "common/note.S"
arch/x86/entry/vdso/vdso64/vclock_gettime.c new file +1 -0
@@ -0,0 +1 @@ +#include "common/vclock_gettime.c"
arch/x86/entry/vdso/vdso64/vdso64.lds.S renamed from arch/x86/entry/vdso/vdso.lds.S +1 -1
@@ -9,7 +9,7 @@ #define BUILD_VDSO64 -#include "vdso-layout.lds.S" +#include "common/vdso-layout.lds.S" /* * This controls what userland symbols we export from the vDSO.
arch/x86/entry/vdso/vdso64/vdsox32.lds.S renamed from arch/x86/entry/vdso/vdsox32.lds.S +1 -1
@@ -9,7 +9,7 @@ #define BUILD_VDSOX32 -#include "vdso-layout.lds.S" +#include "common/vdso-layout.lds.S" /* * This controls what userland symbols we export from the vDSO.
arch/x86/entry/vdso/vdso64/vgetcpu.c new file +1 -0
@@ -0,0 +1 @@ +#include "common/vgetcpu.c"
arch/x86/entry/vdso/vdso64/vgetrandom-chacha.S renamed from arch/x86/entry/vdso/vgetrandom-chacha.S +0 -0
arch/x86/entry/vdso/vdso64/vgetrandom.c renamed from arch/x86/entry/vdso/vgetrandom.c +1 -1
@@ -4,7 +4,7 @@*/ #include <linux/types.h> -#include "../../../../lib/vdso/getrandom.c" +#include "lib/vdso/getrandom.c" ssize_t __vdso_getrandom (void *buffer,size_t len,unsigned int flags,void *opaque_state,size_t opaque_len) {
arch/x86/entry/vdso/vdso64/vsgx.S renamed from arch/x86/entry/vdso/vsgx.S +0 -0
Signature · 1 line
2.52.0
Re: [PATCH v4.1 03/10] x86/entry/vdso: refactor the vdso build
On Tue, 6 Jan 2026 13:18:36 -0800 "H. Peter Anvin" <[email protected]> wrote:
Hi everyone,
I ran the tip master branch through my AI review prompts and this one was flagged. These look right to me, apologies if it's noise:
Quoted 5 lines
arch/x86/entry/vdso/common/Makefile.include new file +0
-0
[ ... ]
arch/x86/entry/vdso/common/Makefile.include line 26
Quoted 6 lines
arch/x86/entry/vdso/common/Makefile.include new file +6
-0+# +# Options from KBUILD_[AC]FLAGS that should *NOT* be kept +# +flags-remove-y += \ + -D__KERNEL__ -mcmodel=kernel -mregparm=3 \ + -fno-pic -fno-PIC -fno-pie fno-PIE \
^^^^^^^
Should this be "-fno-PIE" with the leading dash? The other flags in this line all have the dash prefix, but "fno-PIE" is missing it. Without the dash, filter-out won't match the actual compiler flag.
arch/x86/entry/vdso/common/Makefile.include line 28
Quoted 2 lines
arch/x86/entry/vdso/common/Makefile.include new file +2
-0+ -mfentry -pg \ + $(RANDSTRUCT_CFLAGS) $(GCC_PLUGIN_CFLAGS) $(KSTACK_ERASE_CFLAGS) \
^^^^^^^^^^^^^^^^^
Is GCC_PLUGIN_CFLAGS the right variable name here? The kbuild system defines GCC_PLUGINS_CFLAGS (with an 's') in scripts/Makefile.gcc-plugins. Other vDSO Makefiles (arm64, sparc, arm) use GCC_PLUGINS_CFLAGS. Without the fix, gcc plugin flags won't be filtered out when CONFIG_GCC_PLUGINS=y.
Quoted 2 lines
arch/x86/entry/vdso/common/Makefile.include new file +2
-0+ $(RETPOLINE_CFLAGS) $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) \ + $(PADDING_CFLAGS)
[ ... ]
arch/x86/entry/vdso/common/Makefile.include line 48
Quoted 3 lines
arch/x86/entry/vdso/common/Makefile.include new file +3
-0+# Reset cf protections enabled by compiler default +flags-y += $(call cc-option, -fcf-protection=none) +flags-$(X86_USER_SHADOW_STACK) += $(call cc-option, -fcf-protection=return)
^^^^^^^^^^^^^^^^^^^^^
Should this be CONFIG_X86_USER_SHADOW_STACK? The Kconfig symbol is defined as "config X86_USER_SHADOW_STACK" and kbuild exposes it as a CONFIG_ prefixed variable. The next lines show the pattern:
arch/x86/entry/vdso/common/Makefile.include line 52
Quoted 4 lines
arch/x86/entry/vdso/common/Makefile.include new file +4
-0+# When user space IBT is supported, enable this. +# flags-$(CONFIG_USER_IBT) += $(call cc-option, -fcf-protection=branch) + +flags-$(CONFIG_MITIGATION_RETPOLINE) += $(RETPOLINE_VDSO_CFLAGS)
Both of these correctly use the CONFIG_ prefix. Without it, the shadow stack cf-protection flag will never be added to vDSO builds even when CONFIG_X86_USER_SHADOW_STACK=y.
H. Peter Anvin
maintainerPATCH v4 4/10 x86/entry/vdso32: don't rely on int80_landing_pad for adjusting ip
In reply to H. Peter AnvinThere is no fundamental reason to use the int80_landing_pad symbol to adjust ip when moving the vdso. If ip falls within the vdso, and the vdso is moved, we should change the ip accordingly, regardless of mode or location within the vdso. This *currently* can only happen on 32 bits, but there isn't any reason not to do so generically.
Note that if this is ever possible from a vdso-internal call, then the user space stack will also needed to be adjusted (as well as the shadow stack, if enabled.) Fortunately this is not currently the case.
At the moment, we don't even consider other threads when moving the vdso. The assumption is that it is only used by process freeze/thaw for migration, where this is not an issue.
--- arch/x86/entry/vdso/vma.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
arch/x86/entry/vdso/vma.c +6 -10
@@ -65,16 +65,12 @@ static vm_fault_t vdso_fault(const struct vm_special_mapping *sm,static void vdso_fix_landing (const struct vdso_image *image,struct vm_area_struct *new_vma) { -if (in_ia32_syscall () && image == &vdso32_image) { -struct pt_regs *regs =current_pt_regs (); -unsigned long vdso_land = image->sym_int80_landing_pad; -unsigned long old_land_addr = vdso_land + - (unsigned long )current->mm->context.vdso; - -/* Fixing userspace landing - look at do_fast_syscall_32 */ -if (regs->ip == old_land_addr) - regs->ip = new_vma->vm_start + vdso_land; - } +struct pt_regs *regs =current_pt_regs (); +unsigned long ipoffset = regs->ip - + (unsigned long )current->mm->context.vdso; + +if (ipoffset < image->size) + regs->ip = new_vma->vm_start + ipoffset; } static int vdso_mremap(const struct vm_special_mapping *sm,
Signature · 1 line
2.52.0
H. Peter Anvin
maintainerPATCH v4 5/10 x86/entry/vdso32: remove SYSCALL_ENTER_KERNEL macro in sigreturn.S
In reply to H. Peter AnvinA macro SYSCALL_ENTER_KERNEL was defined in sigreturn.S, with the ability of overriding it. The override capability, however, is not used anywhere, and the macro name is potentially confusing because it seems to imply that sysenter/syscall could be used here, which is NOT
true: the sigreturn system calls MUST use int $0x80.
--- arch/x86/entry/vdso/vdso32/sigreturn.S | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
arch/x86/entry/vdso/vdso32/sigreturn.S +2 -6
@@ -3,10 +3,6 @@ #include <asm/unistd_32.h> #include <asm/asm-offsets.h> -#ifndef SYSCALL_ENTER_KERNEL -#define SYSCALL_ENTER_KERNEL int $0x80 -#endif - .text .globl __kernel_sigreturn .type __kernel_sigreturn,@function
@@ -16,7 +12,7 @@ __kernel_sigreturn: .LSTART_sigreturn: popl %eax /* XXX does this mean it needs unwind info? */ movl $__NR_sigreturn, %eax - SYSCALL_ENTER_KERNEL + int $0x80 .LEND_sigreturn: SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) nop
@@ -28,7 +24,7 @@ SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) __kernel_rt_sigreturn: .LSTART_rt_sigreturn: movl $__NR_rt_sigreturn, %eax - SYSCALL_ENTER_KERNEL + int $0x80 .LEND_rt_sigreturn: SYM_INNER_LABEL(vdso32_rt_sigreturn_landing_pad, SYM_L_GLOBAL) nop
Signature · 1 line
2.52.0
H. Peter Anvin
maintainerPATCH v4 6/10 x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
In reply to H. Peter AnvinThe vdso32 sigreturn.S contains open-coded DWARF bytecode, which includes a hack for gdb to not try to step back to a previous call instruction when backtracing from a signal handler.
Neither of those are necessary anymore: the backtracing issue is handled by ".cfi_entry simple" and ".cfi_signal_frame", both of which have been supported for a very long time now, which allows the remaining frame to be built using regular .cfi annotations.
Add a few more register offsets to the signal frame just for good measure.
Replace the nop on fallthrough of the system call (which should never, ever happen) with a ud2a trap.
--- arch/x86/entry/vdso/vdso32/sigreturn.S | 146 ++++++------------------- arch/x86/include/asm/dwarf2.h | 1 + arch/x86/kernel/asm-offsets.c | 6 + 3 files changed, 39 insertions(+), 114 deletions(-)
arch/x86/entry/vdso/vdso32/sigreturn.S +32 -114
@@ -1,136 +1,54 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include <linux/linkage.h> #include <asm/unistd_32.h> +#include <asm/dwarf2.h> #include <asm/asm-offsets.h> +.macro STARTPROC_SIGNAL_FRAME sc + CFI_STARTPROC simple + CFI_SIGNAL_FRAME + /* -4 as pretcode has already been popped */ + CFI_DEF_CFA esp, \sc - 4 + CFI_OFFSET eip, IA32_SIGCONTEXT_ip + CFI_OFFSET eax, IA32_SIGCONTEXT_ax + CFI_OFFSET ebx, IA32_SIGCONTEXT_bx + CFI_OFFSET ecx, IA32_SIGCONTEXT_cx + CFI_OFFSET edx, IA32_SIGCONTEXT_dx + CFI_OFFSET esp, IA32_SIGCONTEXT_sp + CFI_OFFSET ebp, IA32_SIGCONTEXT_bp + CFI_OFFSET esi, IA32_SIGCONTEXT_si + CFI_OFFSET edi, IA32_SIGCONTEXT_di + CFI_OFFSET es, IA32_SIGCONTEXT_es + CFI_OFFSET cs, IA32_SIGCONTEXT_cs + CFI_OFFSET ss, IA32_SIGCONTEXT_ss + CFI_OFFSET ds, IA32_SIGCONTEXT_ds + CFI_OFFSET eflags, IA32_SIGCONTEXT_flags +.endm + .text .globl __kernel_sigreturn .type __kernel_sigreturn,@function - nop /* this guy is needed for .LSTARTFDEDLSI1 below (watch for HACK) */ ALIGN __kernel_sigreturn: -.LSTART_sigreturn: - popl %eax /* XXX does this mean it needs unwind info? */ + STARTPROC_SIGNAL_FRAME IA32_SIGFRAME_sigcontext + popl %eax + CFI_ADJUST_CFA_OFFSET -4 movl $__NR_sigreturn, %eax int $0x80 -.LEND_sigreturn: SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) - nop - .size __kernel_sigreturn,.-.LSTART_sigreturn + ud2a + CFI_ENDPROC + .size __kernel_sigreturn,.-__kernel_sigreturn .globl __kernel_rt_sigreturn .type __kernel_rt_sigreturn,@function ALIGN __kernel_rt_sigreturn: -.LSTART_rt_sigreturn: + STARTPROC_SIGNAL_FRAME IA32_RT_SIGFRAME_sigcontext movl $__NR_rt_sigreturn, %eax int $0x80 -.LEND_rt_sigreturn: SYM_INNER_LABEL(vdso32_rt_sigreturn_landing_pad, SYM_L_GLOBAL) - nop - .size __kernel_rt_sigreturn,.-.LSTART_rt_sigreturn - .previous - - .section .eh_frame,"a",@progbits -.LSTARTFRAMEDLSI1: - .long .LENDCIEDLSI1-.LSTARTCIEDLSI1 -.LSTARTCIEDLSI1: - .long 0 /* CIE ID */ - .byte 1 /* Version number */ - .string "zRS" /* NUL-terminated augmentation string */ - .uleb128 1 /* Code alignment factor */ - .sleb128 -4 /* Data alignment factor */ - .byte 8 /* Return address register column */ - .uleb128 1 /* Augmentation value length */ - .byte 0x1b /* DW_EH_PE_pcrel|DW_EH_PE_sdata4. */ - .byte 0 /* DW_CFA_nop */ - .align 4 -.LENDCIEDLSI1: - .long .LENDFDEDLSI1-.LSTARTFDEDLSI1 /* Length FDE */ -.LSTARTFDEDLSI1: - .long .LSTARTFDEDLSI1-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: The dwarf2 unwind routines will subtract 1 from the - return address to get an address in the middle of the - presumed call instruction. Since we didn't get here via - a call, we need to include the nop before the real start - to make up for it. */ - .long .LSTART_sigreturn-1-. /* PC-relative start address */ - .long .LEND_sigreturn-.LSTART_sigreturn+1 - .uleb128 0 /* Augmentation */ - /* What follows are the instructions for the table generation. - We record the locations of each register saved. This is - complicated by the fact that the "CFA" is always assumed to - be the value of the stack pointer in the caller. This means - that we must define the CFA of this body of code to be the - saved value of the stack pointer in the sigcontext. Which - also means that there is no fixed relation to the other - saved registers, which means that we must use DW_CFA_expression - to compute their addresses. It also means that when we - adjust the stack with the popl, we have to do it all over again. */ - -#define do_cfa_expr(offset) \ - .byte 0x0f; /* DW_CFA_def_cfa_expression */ \ - .uleb128 1f-0f; /* length */ \ -0: .byte 0x74; /* DW_OP_breg4 */ \ - .sleb128 offset; /* offset */ \ - .byte 0x06; /* DW_OP_deref */ \ -1: - -#define do_expr(regno, offset) \ - .byte 0x10; /* DW_CFA_expression */ \ - .uleb128 regno; /* regno */ \ - .uleb128 1f-0f; /* length */ \ -0: .byte 0x74; /* DW_OP_breg4 */ \ - .sleb128 offset; /* offset */ \ -1: - - do_cfa_expr(IA32_SIGCONTEXT_sp+4) - do_expr(0, IA32_SIGCONTEXT_ax+4) - do_expr(1, IA32_SIGCONTEXT_cx+4) - do_expr(2, IA32_SIGCONTEXT_dx+4) - do_expr(3, IA32_SIGCONTEXT_bx+4) - do_expr(5, IA32_SIGCONTEXT_bp+4) - do_expr(6, IA32_SIGCONTEXT_si+4) - do_expr(7, IA32_SIGCONTEXT_di+4) - do_expr(8, IA32_SIGCONTEXT_ip+4) - - .byte 0x42 /* DW_CFA_advance_loc 2 -- nop; popl eax. */ - - do_cfa_expr(IA32_SIGCONTEXT_sp) - do_expr(0, IA32_SIGCONTEXT_ax) - do_expr(1, IA32_SIGCONTEXT_cx) - do_expr(2, IA32_SIGCONTEXT_dx) - do_expr(3, IA32_SIGCONTEXT_bx) - do_expr(5, IA32_SIGCONTEXT_bp) - do_expr(6, IA32_SIGCONTEXT_si) - do_expr(7, IA32_SIGCONTEXT_di) - do_expr(8, IA32_SIGCONTEXT_ip) - - .align 4 -.LENDFDEDLSI1: - - .long .LENDFDEDLSI2-.LSTARTFDEDLSI2 /* Length FDE */ -.LSTARTFDEDLSI2: - .long .LSTARTFDEDLSI2-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: See above wrt unwind library assumptions. */ - .long .LSTART_rt_sigreturn-1-. /* PC-relative start address */ - .long .LEND_rt_sigreturn-.LSTART_rt_sigreturn+1 - .uleb128 0 /* Augmentation */ - /* What follows are the instructions for the table generation. - We record the locations of each register saved. This is - slightly less complicated than the above, since we don't - modify the stack pointer in the process. */ - - do_cfa_expr(IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_sp) - do_expr(0, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_ax) - do_expr(1, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_cx) - do_expr(2, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_dx) - do_expr(3, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_bx) - do_expr(5, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_bp) - do_expr(6, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_si) - do_expr(7, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_di) - do_expr(8, IA32_RT_SIGFRAME_sigcontext-4 + IA32_SIGCONTEXT_ip) - - .align 4 -.LENDFDEDLSI2: + ud2a + CFI_ENDPROC + .size __kernel_rt_sigreturn,.-__kernel_rt_sigreturn .previous
arch/x86/include/asm/dwarf2.h +1 -0
@@ -20,6 +20,7 @@#define CFI_RESTORE_STATE .cfi_restore_state#define CFI_UNDEFINED .cfi_undefined#define CFI_ESCAPE .cfi_escape +#define CFI_SIGNAL_FRAME .cfi_signal_frame#ifndef BUILD_VDSO /*
arch/x86/kernel/asm-offsets.c +6 -0
@@ -63,8 +63,14 @@ static void __used common(void)OFFSET (IA32_SIGCONTEXT_bp, sigcontext_32, bp);OFFSET (IA32_SIGCONTEXT_sp, sigcontext_32, sp);OFFSET (IA32_SIGCONTEXT_ip, sigcontext_32, ip); +OFFSET (IA32_SIGCONTEXT_es, sigcontext_32, es); +OFFSET (IA32_SIGCONTEXT_cs, sigcontext_32, cs); +OFFSET (IA32_SIGCONTEXT_ss, sigcontext_32, ss); +OFFSET (IA32_SIGCONTEXT_ds, sigcontext_32, ds); +OFFSET (IA32_SIGCONTEXT_flags, sigcontext_32, flags);BLANK (); +OFFSET (IA32_SIGFRAME_sigcontext, sigframe_ia32, sc);OFFSET (IA32_RT_SIGFRAME_sigcontext, rt_sigframe_ia32, uc.uc_mcontext);#endif
Signature · 1 line
2.52.0
Re: [PATCH v4.1 06/10] x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
Hello Peter!
Quoted 8 lines · H. Peter Anvin
The vdso32
sigreturn.Scontains open-coded DWARF bytecode, which includes a hack for gdb to not try to step back to a previous call instruction when backtracing from a signal handler.Neither of those are necessary anymore: the backtracing issue is handled by ".cfi_entry simple" and ".cfi_signal_frame", both of which have been supported for a very long time now, which allows the remaining frame to be built using regular .cfi annotations.
Hopefully Glibc developers will do something similar for x86-64 __restore_rt() in Glibc sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c.
Quoted 22 lines
Add a few more register offsets to the signal frame just for good measure.
Replace the nop on fallthrough of the system call (which should never, ever happen) with a ud2a trap.
arch/x86/entry/vdso/vdso32/sigreturn.S +3
-4.text .globl __kernel_sigreturn .type __kernel_sigreturn,@function - nop /* this guy is needed for .LSTARTFDEDLSI1 below (watch for HACK) */ ALIGN __kernel_sigreturn: -.LSTART_sigreturn: - popl %eax /* XXX does this mean it needs unwind info? */ + STARTPROC_SIGNAL_FRAME IA32_SIGFRAME_sigcontext + popl %eax + CFI_ADJUST_CFA_OFFSET -4 movl $__NR_sigreturn, %eax int $0x80 -.LEND_sigreturn:
...
Quoted 9 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +1
-2.globl __kernel_rt_sigreturn .type __kernel_rt_sigreturn,@function ALIGN __kernel_rt_sigreturn: -.LSTART_rt_sigreturn: + STARTPROC_SIGNAL_FRAME IA32_RT_SIGFRAME_sigcontext movl $__NR_rt_sigreturn, %eax int $0x80 -.LEND_rt_sigreturn:
...
arch/x86/entry/vdso/vdso32/sigreturn.S line 40
Quoted 7 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-7- .section .eh_frame,"a",@progbits -.LSTARTFRAMEDLSI1: - .long .LENDCIEDLSI1-.LSTARTCIEDLSI1 -.LSTARTCIEDLSI1: - .long 0 /* CIE ID */ - .byte 1 /* Version number */ - .string "zRS" /* NUL-terminated augmentation string */
Note that the "S" in "zRS" is the signal frame indication.
Quoted 7 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-7- .uleb128 1 /* Code alignment factor */ - .sleb128 -4 /* Data alignment factor */ - .byte 8 /* Return address register column */ - .uleb128 1 /* Augmentation value length */ - .byte 0x1b /* DW_EH_PE_pcrel|DW_EH_PE_sdata4. */ - .byte 0 /* DW_CFA_nop */ - .align 4
arch/x86/entry/vdso/vdso32/sigreturn.S line 57
Quoted 10 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-10-.LENDCIEDLSI1: - .long .LENDFDEDLSI1-.LSTARTFDEDLSI1 /* Length FDE */ -.LSTARTFDEDLSI1: - .long .LSTARTFDEDLSI1-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: The dwarf2 unwind routines will subtract 1 from the - return address to get an address in the middle of the - presumed call instruction. Since we didn't get here via - a call, we need to include the nop before the real start - to make up for it. */ - .long .LSTART_sigreturn-1-. /* PC-relative start address */
Your version does no longer have this nop nor does the FDE start one byte earlier. Isn't that required for unwinders any longer? See excerpt from dumped DWARF and disassembly for __kernel_sigreturn() below.
Quoted 2 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-2- .long .LEND_sigreturn-.LSTART_sigreturn+1 - .uleb128 0 /* Augmentation */
...
arch/x86/entry/vdso/vdso32/sigreturn.S line 116
Quoted 8 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-8- .align 4 -.LENDFDEDLSI1: - - .long .LENDFDEDLSI2-.LSTARTFDEDLSI2 /* Length FDE */ -.LSTARTFDEDLSI2: - .long .LSTARTFDEDLSI2-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: See above wrt unwind library assumptions. */ - .long .LSTART_rt_sigreturn-1-. /* PC-relative start address */
Ditto.
arch/x86/entry/vdso/vdso32/sigreturn.S line 118
Quoted 2 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-2- .long .LEND_rt_sigreturn-.LSTART_rt_sigreturn+1 - .uleb128 0 /* Augmentation */
Excerpt from dump of DWARF and disassembly with your patch:
$ objdump -d -Wf arch/x86/entry/vdso/vdso32/vdso32.so.dbg ... 000001cc 0000003c 00000000 CIE <-- CIE for __kernel_sigreturn Version: 1 Augmentation: "zRS" Code alignment factor: 1 Data alignment factor: -4 Return address column: 8 Augmentation data: 1b DW_CFA_def_cfa: r4 (esp) ofs 4 DW_CFA_offset_extended_sf: r8 (eip) at cfa+56 DW_CFA_offset_extended_sf: r0 (eax) at cfa+44 DW_CFA_offset_extended_sf: r3 (ebx) at cfa+32 DW_CFA_offset_extended_sf: r1 (ecx) at cfa+40 DW_CFA_offset_extended_sf: r2 (edx) at cfa+36 DW_CFA_offset_extended_sf: r4 (esp) at cfa+28 DW_CFA_offset_extended_sf: r5 (ebp) at cfa+24 DW_CFA_offset_extended_sf: r6 (esi) at cfa+20 DW_CFA_offset_extended_sf: r7 (edi) at cfa+16 DW_CFA_offset_extended_sf: r40 (es) at cfa+8 DW_CFA_offset_extended_sf: r41 (cs) at cfa+60 DW_CFA_offset_extended_sf: r42 (ss) at cfa+72 DW_CFA_offset_extended_sf: r43 (ds) at cfa+12 DW_CFA_offset_extended_sf: r9 (eflags) at cfa+64 DW_CFA_nop
0000020c 00000010 00000044 FDE cie=000001cc pc=00001a40..00001a4a <-- FDE for __kernel_sigreturn
DW_CFA_advance_loc: 1 to 00001a41
DW_CFA_def_cfa_offset: 0
[ The FDE covers the range [1a40..1a4a[. Previously it would have started one byte earlier (at the nop), so that the range would have been [1a3f..1a4a[. This is/was required for unwinders that always subtract one from the unwound return address, so that it points into the instruction that invoked the function (e.g. call) instead of behind it, in case it was invoked by a non-returning function. Such an unwinder would now lookup IP=1a3f as belonging to int80_landing_pad (and use the DWARF rules applicable to its last instruction) instead of __kernel_sigreturn (and its rules). Likewise for __kernel_rt_sigreturn. ]
...00001a3c <int80_landing_pad>:
1a3c: 5d pop %ebp
1a3d: 5a pop %edx
1a3e: 59 pop %ecx
1a3f: c3 ret00001a40 <__kernel_sigreturn>:
1a40: 58 pop %eax
1a41: b8 77 00 00 00 mov $0x77,%eax
1a46: cd 80 int $0x8000001a48 <vdso32_sigreturn_landing_pad>:
1a48: 0f 0b ud2
1a4a: 8d b6 00 00 00 00 lea 0x0(%esi),%esiExcerpt without your patch:
$ objdump -d -Wf arch/x86/entry/vdso/vdso32/vdso32.so.dbg ... 000001cc 00000010 00000000 CIE <-- CIE for __kernel_sigreturn and __kernel_rt_sigreturn Version: 1 Augmentation: "zRS" Code alignment factor: 1 Data alignment factor: -4 Return address column: 8 Augmentation data: 1b DW_CFA_nop DW_CFA_nop
000001e0 00000068 00000018 FDE cie=000001cc pc=00001a6f..00001a78 <-- FDE for __kernel_sigreturn
DW_CFA_def_cfa_expression (DW_OP_breg4 (esp): 32; DW_OP_deref)
DW_CFA_expression: r0 (eax) (DW_OP_breg4 (esp): 48)
DW_CFA_expression: r1 (ecx) (DW_OP_breg4 (esp): 44)
DW_CFA_expression: r2 (edx) (DW_OP_breg4 (esp): 40)
DW_CFA_expression: r3 (ebx) (DW_OP_breg4 (esp): 36)
DW_CFA_expression: r5 (ebp) (DW_OP_breg4 (esp): 28)
DW_CFA_expression: r6 (esi) (DW_OP_breg4 (esp): 24)
DW_CFA_expression: r7 (edi) (DW_OP_breg4 (esp): 20)
DW_CFA_expression: r8 (eip) (DW_OP_breg4 (esp): 60)
DW_CFA_advance_loc: 2 to 00001a71
DW_CFA_def_cfa_expression (DW_OP_breg4 (esp): 28; DW_OP_deref)
DW_CFA_expression: r0 (eax) (DW_OP_breg4 (esp): 44)
DW_CFA_expression: r1 (ecx) (DW_OP_breg4 (esp): 40)
DW_CFA_expression: r2 (edx) (DW_OP_breg4 (esp): 36)
DW_CFA_expression: r3 (ebx) (DW_OP_breg4 (esp): 32)
DW_CFA_expression: r5 (ebp) (DW_OP_breg4 (esp): 24)
DW_CFA_expression: r6 (esi) (DW_OP_breg4 (esp): 20)
DW_CFA_expression: r7 (edi) (DW_OP_breg4 (esp): 16)
DW_CFA_expression: r8 (eip) (DW_OP_breg4 (esp): 56)
[ See how the FDE for __kernel_sigreturn covers the range [1a6f..1a78[. An unwinder that always subtracts one from the return address would lookup IP=1a6f as belonging to __kernel_sigreturn (and use the DWARF rules applicable to the nop preceeding its symbol). Likewise for __kernel_rt_sigreturn. Or is that no longer true? ]
...00001a5c <int80_landing_pad>:
1a5c: 5d pop %ebp
1a5d: 5a pop %edx
1a5e: 59 pop %ecx
1a5f: c3 ret
1a60: 90 nop
1a61: 8d b4 26 00 00 00 00 lea 0x0(%esi,%eiz,1),%esi
1a68: 2e 8d b4 26 00 00 00 lea %cs:0x0(%esi,%eiz,1),%esi
1a6f: 0000001a70 <__kernel_sigreturn>:
1a70: 58 pop %eax
1a71: b8 77 00 00 00 mov $0x77,%eax
1a76: cd 80 int $0x80Thanks and regards,
Jens
Signature · 6 lines
Jens Remus Linux on Z Development (D3303) [email protected] / [email protected] IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294 IBM Data Privacy Statement: https://www.ibm.com/privacy/
H. Peter Anvin
maintainerRe: [PATCH v4.1 06/10] x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
Quoted 215 lines · Jens Remus
Hello Peter!
Quoted 8 lines · H. Peter Anvin
The vdso32
sigreturn.Scontains open-coded DWARF bytecode, which includes a hack for gdb to not try to step back to a previous call instruction when backtracing from a signal handler.Neither of those are necessary anymore: the backtracing issue is handled by ".cfi_entry simple" and ".cfi_signal_frame", both of which have been supported for a very long time now, which allows the remaining frame to be built using regular .cfi annotations.
Hopefully Glibc developers will do something similar for x86-64
__restore_rt()in Glibcsysdeps/unix/sysv/linux/x86_64/libc_sigaction.c.Quoted 22 lines
Add a few more register offsets to the signal frame just for good measure.
Replace the nop on fallthrough of the system call (which should never, ever happen) with a ud2a trap.
arch/x86/entry/vdso/vdso32/sigreturn.S +3
-4.text .globl __kernel_sigreturn .type __kernel_sigreturn,@function - nop /* this guy is needed for .LSTARTFDEDLSI1 below (watch for HACK) */ ALIGN __kernel_sigreturn: -.LSTART_sigreturn: - popl %eax /* XXX does this mean it needs unwind info? */ + STARTPROC_SIGNAL_FRAME IA32_SIGFRAME_sigcontext + popl %eax + CFI_ADJUST_CFA_OFFSET -4 movl $__NR_sigreturn, %eax int $0x80 -.LEND_sigreturn:...
Quoted 9 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +1
-2.globl __kernel_rt_sigreturn .type __kernel_rt_sigreturn,@function ALIGN __kernel_rt_sigreturn: -.LSTART_rt_sigreturn: + STARTPROC_SIGNAL_FRAME IA32_RT_SIGFRAME_sigcontext movl $__NR_rt_sigreturn, %eax int $0x80 -.LEND_rt_sigreturn:...
arch/x86/entry/vdso/vdso32/sigreturn.S line 40
Quoted 7 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-7- .section .eh_frame,"a",@progbits -.LSTARTFRAMEDLSI1: - .long .LENDCIEDLSI1-.LSTARTCIEDLSI1 -.LSTARTCIEDLSI1: - .long 0 /* CIE ID */ - .byte 1 /* Version number */ - .string "zRS" /* NUL-terminated augmentation string */Note that the "S" in "zRS" is the signal frame indication.
Quoted 7 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-7- .uleb128 1 /* Code alignment factor */ - .sleb128 -4 /* Data alignment factor */ - .byte 8 /* Return address register column */ - .uleb128 1 /* Augmentation value length */ - .byte 0x1b /* DW_EH_PE_pcrel|DW_EH_PE_sdata4. */ - .byte 0 /* DW_CFA_nop */ - .align 4arch/x86/entry/vdso/vdso32/sigreturn.S line 57
Quoted 10 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-10-.LENDCIEDLSI1: - .long .LENDFDEDLSI1-.LSTARTFDEDLSI1 /* Length FDE */ -.LSTARTFDEDLSI1: - .long .LSTARTFDEDLSI1-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: The dwarf2 unwind routines will subtract 1 from the - return address to get an address in the middle of the - presumed call instruction. Since we didn't get here via - a call, we need to include the nop before the real start - to make up for it. */ - .long .LSTART_sigreturn-1-. /* PC-relative start address */Your version does no longer have this nop nor does the FDE start one byte earlier. Isn't that required for unwinders any longer? See excerpt from dumped DWARF and disassembly for
__kernel_sigreturn()below.Quoted 2 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-2- .long .LEND_sigreturn-.LSTART_sigreturn+1 - .uleb128 0 /* Augmentation */...
arch/x86/entry/vdso/vdso32/sigreturn.S line 116
Quoted 8 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-8- .align 4 -.LENDFDEDLSI1: - - .long .LENDFDEDLSI2-.LSTARTFDEDLSI2 /* Length FDE */ -.LSTARTFDEDLSI2: - .long .LSTARTFDEDLSI2-.LSTARTFRAMEDLSI1 /* CIE pointer */ - /* HACK: See above wrt unwind library assumptions. */ - .long .LSTART_rt_sigreturn-1-. /* PC-relative start address */Ditto.
arch/x86/entry/vdso/vdso32/sigreturn.S line 118
Quoted 2 lines
arch/x86/entry/vdso/vdso32/sigreturn.S +0
-2- .long .LEND_rt_sigreturn-.LSTART_rt_sigreturn+1 - .uleb128 0 /* Augmentation */Excerpt from dump of DWARF and disassembly with your patch:
$ objdump -d -Wf arch/x86/entry/vdso/vdso32/vdso32.so.dbg ... 000001cc 0000003c 00000000 CIE <-- CIE for __kernel_sigreturn Version: 1 Augmentation: "zRS" Code alignment factor: 1 Data alignment factor: -4 Return address column: 8 Augmentation data: 1b DW_CFA_def_cfa: r4 (esp) ofs 4 DW_CFA_offset_extended_sf: r8 (eip) at cfa+56 DW_CFA_offset_extended_sf: r0 (eax) at cfa+44 DW_CFA_offset_extended_sf: r3 (ebx) at cfa+32 DW_CFA_offset_extended_sf: r1 (ecx) at cfa+40 DW_CFA_offset_extended_sf: r2 (edx) at cfa+36 DW_CFA_offset_extended_sf: r4 (esp) at cfa+28 DW_CFA_offset_extended_sf: r5 (ebp) at cfa+24 DW_CFA_offset_extended_sf: r6 (esi) at cfa+20 DW_CFA_offset_extended_sf: r7 (edi) at cfa+16 DW_CFA_offset_extended_sf: r40 (es) at cfa+8 DW_CFA_offset_extended_sf: r41 (cs) at cfa+60 DW_CFA_offset_extended_sf: r42 (ss) at cfa+72 DW_CFA_offset_extended_sf: r43 (ds) at cfa+12 DW_CFA_offset_extended_sf: r9 (eflags) at cfa+64 DW_CFA_nop
0000020c00000010 00000044 FDE cie=000001cc pc=00001a40..00001a4a <-- FDE for__kernel_sigreturn
DW_CFA_advance_loc: 1 to00001a41
DW_CFA_def_cfa_offset: 0[ The FDE covers the range [1a40..1a4a[. Previously it would have started one byte earlier (at the nop), so that the range would have been [1a3f..1a4a[. This is/was required for unwinders that always subtract one from the unwound return address, so that it points into the instruction that invoked the function (e.g. call) instead of behind it, in case it was invoked by a non-returning function. Such an unwinder would now lookup IP=1a3f as belonging to
int80_landing_pad(and use the DWARF rules applicable to its last instruction) instead of__kernel_sigreturn(and its rules). Likewise for__kernel_rt_sigreturn. ]...
00001a3c<int80_landing_pad>:1a3c: 5d pop %ebp 1a3d: 5a pop %edx 1a3e: 59 pop %ecx 1a3f: c3 ret
00001a40<__kernel_sigreturn>:1a40: 58 pop %eax 1a41: b8 77 00 00 00 mov $0x77,%eax 1a46: cd 80 int $0x80
00001a48<vdso32_sigreturn_landing_pad>:1a48: 0f 0b ud2 1a4a: 8d b6 00 00 00 00 lea 0x0(%esi),%esiExcerpt without your patch:
$ objdump -d -Wf arch/x86/entry/vdso/vdso32/vdso32.so.dbg ... 000001cc 00000010 00000000 CIE <-- CIE for __kernel_sigreturn and __kernel_rt_sigreturn Version: 1 Augmentation: "zRS" Code alignment factor: 1 Data alignment factor: -4 Return address column: 8 Augmentation data: 1b DW_CFA_nop DW_CFA_nop
000001e000000068 00000018 FDE cie=000001cc pc=00001a6f..00001a78 <-- FDE for__kernel_sigreturn
DW_CFA_def_cfa_expression(DW_OP_breg4(esp): 32;DW_OP_deref)
DW_CFA_expression: r0 (eax) (DW_OP_breg4(esp): 48)
DW_CFA_expression: r1 (ecx) (DW_OP_breg4(esp): 44)
DW_CFA_expression: r2 (edx) (DW_OP_breg4(esp): 40)
DW_CFA_expression: r3 (ebx) (DW_OP_breg4(esp): 36)
DW_CFA_expression: r5 (ebp) (DW_OP_breg4(esp): 28)
DW_CFA_expression: r6 (esi) (DW_OP_breg4(esp): 24)
DW_CFA_expression: r7 (edi) (DW_OP_breg4(esp): 20)
DW_CFA_expression: r8 (eip) (DW_OP_breg4(esp): 60)
DW_CFA_advance_loc: 2 to00001a71
DW_CFA_def_cfa_expression(DW_OP_breg4(esp): 28;DW_OP_deref)
DW_CFA_expression: r0 (eax) (DW_OP_breg4(esp): 44)
DW_CFA_expression: r1 (ecx) (DW_OP_breg4(esp): 40)
DW_CFA_expression: r2 (edx) (DW_OP_breg4(esp): 36)
DW_CFA_expression: r3 (ebx) (DW_OP_breg4(esp): 32)
DW_CFA_expression: r5 (ebp) (DW_OP_breg4(esp): 24)
DW_CFA_expression: r6 (esi) (DW_OP_breg4(esp): 20)
DW_CFA_expression: r7 (edi) (DW_OP_breg4(esp): 16)
DW_CFA_expression: r8 (eip) (DW_OP_breg4(esp): 56)[ See how the FDE for
__kernel_sigreturncovers the range [1a6f..1a78[. An unwinder that always subtracts one from the return address would lookup IP=1a6f as belonging to__kernel_sigreturn(and use the DWARF rules applicable to the nop preceeding its symbol). Likewise for__kernel_rt_sigreturn. Or is that no longer true? ]...
00001a5c<int80_landing_pad>:1a5c: 5d pop %ebp 1a5d: 5a pop %edx 1a5e: 59 pop %ecx 1a5f: c3 ret 1a60: 90 nop 1a61: 8d b4 26 00 00 00 00 lea 0x0(%esi,%eiz,1),%esi 1a68: 2e 8d b4 26 00 00 00 lea %cs:0x0(%esi,%eiz,1),%esi 1a6f: 00
Quoted 7 lines
00001a70<__kernel_sigreturn>:1a70: 58 pop %eax 1a71: b8 77 00 00 00 mov $0x77,%eax 1a76: cd 80 int $0x80Thanks and regards,
Jens
That hack dates back from before the signal frame extension. It is no longer necessary.
Re: [PATCH v4.1 06/10] x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
Quoted 2 lines · H. Peter Anvin
That hack dates back from before the signal frame extension. It is no longer necessary.
Unfortunately at least it seems libgcc unwinder does not handle the signal frame extension properly. The code reads:
fde = _Unwind_Find_FDE (context->ra + _Unwind_IsSignalFrame (context) - 1,
&context->bases);
if (fde == NULL)
{ #ifdef MD_FALLBACK_FRAME_STATE_FOR
/* Couldn't find frame unwind info for this function. Try a
target-specific fallback mechanism. This will necessarily not provide a personality routine or LSDA. */
return MD_FALLBACK_FRAME_STATE_FOR (context, fs);
#else
return _URC_END_OF_STACK;
#endif
} fs->pc = context->bases.func; cie = get_cie (fde); insn = extract_cie_info (cie, context, fs);
Thus, it indeed attempts to avoid subtracting 1 for a signal frame, but ... _Unwind_IsSignalFrame (context) actually extracts a flag in context which will only be raised up by extract_cie_info.
Or am I missing something here?
Signature · 1 line
Xi Ruoyao <[email protected]>
H. Peter Anvin
maintainerRe: [PATCH v4.1 06/10] x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
Quoted 23 lines · Xi Ruoyao
Quoted 2 lines · H. Peter Anvin
That hack dates back from before the signal frame extension. It is no longer necessary.
Unfortunately at least it seems libgcc unwinder does not handle the signal frame extension properly. The code reads:
fde =
_Unwind_Find_FDE(context->ra+_Unwind_IsSignalFrame(context) - 1,&context->bases);if (fde == NULL)
{
#ifdefMD_FALLBACK_FRAME_STATE_FOR
/* Couldn't find frame unwind info for this function. Try a
target-specific fallback mechanism. This will necessarily not provide a personality routine or LSDA. */
returnMD_FALLBACK_FRAME_STATE_FOR(context, fs);#else return _URC_END_OF_STACK; #endif }
fs->pc=context->bases.func;
Quoted 9 lines
cie =
get_cie(fde);
insn =extract_cie_info(cie, context, fs);Thus, it indeed attempts to avoid subtracting 1 for a signal frame, but ...
_Unwind_IsSignalFrame(context) actually extracts a flag in context which will only be raised up byextract_cie_info.Or am I missing something here?
Oh, good grief...
How does this possibly work on non-x86 platforms?
Re: [PATCH v4.1 06/10] x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
Quoted 31 lines · H. Peter Anvin
Quoted 30 lines · Xi Ruoyao
Quoted 2 lines · H. Peter Anvin
That hack dates back from before the signal frame extension. It is no longer necessary.
Unfortunately at least it seems libgcc unwinder does not handle the signal frame extension properly. The code reads:
fde =
_Unwind_Find_FDE(context->ra+_Unwind_IsSignalFrame(context) - 1,&context->bases);
if (fde == NULL)
{
#ifdefMD_FALLBACK_FRAME_STATE_FOR
/* Couldn't find frame unwind info for this function. Try a target-specific fallback mechanism. This will necessarily not provide a personality routine or LSDA. */
returnMD_FALLBACK_FRAME_STATE_FOR(context, fs);
#else
return_URC_END_OF_STACK;
#endif
}fs->pc =
context->bases.func;cie =
get_cie(fde);
insn =extract_cie_info(cie, context, fs);Thus, it indeed attempts to avoid subtracting 1 for a signal frame, but ...
_Unwind_IsSignalFrame(context) actually extracts a flag in context which will only be raised up byextract_cie_info.
Quoted 6 lines
Quoted 2 lines
Or am I missing something here?
Oh, good grief...
How does this possibly work on non-x86 platforms?
On ARM64 the vdso does not have eh_frame_hdr at all, on LoongArch eh_frame_hdr is empty (note that an ampty en_frame_hdr is actually buggy and I'm trying to fix it), so _Unwind_Find_FDE returns NULL and libgcc falls back to MD_FALLBACK_FRAME_STATE_FOR, which handles the sigreturn trampoline using some machine-dependant logic.
On RISC-V things are more theatrical: the sigreturn trampoline happens to be at the beginning of the vdso .text section, so after subtracting 1 from the PC, the result is out of the .text section and so not in any FDE. Thus _Unwind_Find_FDE returns NULL and libgcc again falls back to MD_FALLBACK_FRAME_STATE_FOR.
If the RISC-V sigreturn trampoline was not the first in .text, subtracting 1 would cause the PC to be in the FDE of the previous function and then _Unwind_Find_FDE would return that FDE, then RISC-V would have some big trouble.
I've not taken a serious look at other architectures yet.
Signature · 1 line
Xi Ruoyao <[email protected]>
H. Peter Anvin
maintainerRe: [PATCH v4.1 06/10] x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
Quoted 17 lines · Xi Ruoyao
Quoted 4 lines
Oh, good grief...
How does this possibly work on non-x86 platforms?
On ARM64 the vdso does not have
eh_frame_hdrat all, on LoongArcheh_frame_hdris empty (note that an amptyen_frame_hdris actually buggy and I'm trying to fix it), so_Unwind_Find_FDEreturns NULL and libgcc falls back toMD_FALLBACK_FRAME_STATE_FOR, which handles the sigreturn trampoline using some machine-dependant logic.On RISC-V things are more theatrical: the sigreturn trampoline happens to be at the beginning of the vdso .text section, so after subtracting 1 from the PC, the result is out of the .text section and so not in any FDE. Thus
_Unwind_Find_FDEreturns NULL and libgcc again falls back toMD_FALLBACK_FRAME_STATE_FOR.
Quoted 7 lines
If the RISC-V sigreturn trampoline was not the first in .text, subtracting 1 would cause the PC to be in the FDE of the previous function and then
_Unwind_Find_FDEwould return that FDE, then RISC-V would have some big trouble.I've not taken a serious look at other architectures yet.
In other words, it is broken on pretty much all architectures and it ends up using the hacky fallbacks on every single platform...
-hpa
H. Peter Anvin
maintainerRe: [PATCH v4.1 06/10] x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
In reply to Xi RuoyaoQuoted 26 lines · Xi Ruoyao
Quoted 2 lines · H. Peter Anvin
That hack dates back from before the signal frame extension. It is no longer necessary.
Unfortunately at least it seems libgcc unwinder does not handle the signal frame extension properly. The code reads:
fde =
_Unwind_Find_FDE(context->ra+_Unwind_IsSignalFrame(context) - 1,&context->bases); if (fde == NULL) {#ifdef
MD_FALLBACK_FRAME_STATE_FOR
/* Couldn't find frame unwind info for this function. Try a
target-specific fallback mechanism. This will necessarily not provide a personality routine or LSDA. */
returnMD_FALLBACK_FRAME_STATE_FOR(context, fs);#else return _URC_END_OF_STACK; #endif }fs->pc = context->bases.func; cie = get_cie (fde); insn = extract_cie_info (cie, context, fs);
Quoted 6 lines
Thus, it indeed attempts to avoid subtracting 1 for a signal frame, but ...
_Unwind_IsSignalFrame(context) actually extracts a flag in context which will only be raised up byextract_cie_info.Or am I missing something here?
Well, it looks like this might be a non-manifest bug on x86 *IN THIS CASE*, because the fallback handler looks explicitly for the byte sequences:
popl %eax movl $__NR_ia32_sigreturn, %eax int $0x80 movl $__NR_ia32_rt_sigreturn, %eax int $0x80 movq $__NR_x64_rt_sigreturn, %rax /* !!!! */ syscall
... so the vdso patchset is still OK.
Hard-coding a reliance on movq in the 64-bit sequence some serious brain damage...
Could you file a gcc bug on this?
-hpa
Re: [PATCH v4.1 06/10] x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
Quoted 27 lines · H. Peter Anvin
Quoted 26 lines · Xi Ruoyao
Quoted 2 lines · H. Peter Anvin
That hack dates back from before the signal frame extension. It is no longer necessary.
Unfortunately at least it seems libgcc unwinder does not handle the signal frame extension properly. The code reads:
fde =
_Unwind_Find_FDE(context->ra+_Unwind_IsSignalFrame(context) - 1,&context->bases);
if (fde == NULL)
{
#ifdefMD_FALLBACK_FRAME_STATE_FOR
/* Couldn't find frame unwind info for this function. Try a target-specific fallback mechanism. This will necessarily not provide a personality routine or LSDA. */
returnMD_FALLBACK_FRAME_STATE_FOR(context, fs);
#else
return_URC_END_OF_STACK;
#endif
}
fs->pc=context->bases.func;cie =
get_cie(fde);
insn =extract_cie_info(cie, context, fs);
Quoted 8 lines
Quoted 6 lines
Thus, it indeed attempts to avoid subtracting 1 for a signal frame, but ...
_Unwind_IsSignalFrame(context) actually extracts a flag in context which will only be raised up byextract_cie_info.Or am I missing something here?
Well, it looks like this might be a non-manifest bug on x86 *IN THIS CASE*,
But only if after the patch _Unwind_Find_FDE returns NULL here (instead of a FDE of another function). I've not verified it on x86. If glibc tests (specifically misc/tst-sigcontext-get_pc) work fine things should be fine.
/* snip */
Quoted 1 line
Could you file a gcc bug on this?
Will do.
Signature · 1 line
Xi Ruoyao <[email protected]>
H. Peter Anvin
maintainerRe: [PATCH v4.1 06/10] x86/entry/vdso32: remove open-coded DWARF in sigreturn.S
Quoted 8 lines · Xi Ruoyao
Quoted 2 lines
Well, it looks like this might be a non-manifest bug on x86 *IN THIS CASE*,
But only if after the patch
_Unwind_Find_FDEreturns NULL here (instead of a FDE of another function). I've not verified it on x86. If glibc tests (specifically misc/tst-sigcontext-get_pc) work fine things should be fine.
The x86 vdso should be fine because of the alignment padding, but we should probably add an explicit pad before just in case.
What a mess.
-hpa
H. Peter Anvin
maintainerPATCH v4 7/10 x86/entry/vdso: include GNU_PROPERTY and GNU_STACK PHDRs
In reply to H. Peter AnvinCurrently the vdso doesn't include .note.gnu.property or a GNU noexec stack annotation (the -z noexecstack in the linker script is ineffective because we specify PHDRs explicitly.)
The motivation is that the dynamic linker currently do not check these.
However, this is a weak excuse: the vdso*.so are also supposed to be usable at link libraries, and there is no reason why the dynamic linker might not want or need to check these in the future, so add them back in -- it is trivial enough.
Use symbolic constants for the PHDR permission flags.
[ v4: drop unrelated formatting changes ]
[ v4.1: drop one last bogus formatting change (Brian Gerst) ]
--- arch/x86/entry/vdso/common/vdso-layout.lds.S | 36 ++++++++++++-------- 1 file changed, 22 insertions(+), 14 deletions(-)
arch/x86/entry/vdso/common/vdso-layout.lds.S +22 -14
@@ -47,18 +47,18 @@ SECTIONS *(.gnu.linkonce.b.*) } :text - /* - * Discard .note.gnu.property sections which are unused and have - * different alignment requirement from vDSO note sections. - */ - /DISCARD/ : { + .note.gnu.property : { *(.note.gnu.property) - } - .note : { *(.note.*) } :text :note - - .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr - .eh_frame : { KEEP (*(.eh_frame)) } :text + } :text :note :gnu_property + .note : { + *(.note*) + } :text :note + .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr + .eh_frame : { + KEEP (*(.eh_frame)) + *(.eh_frame.*) + } :text /* * Text is well-separated from actual data: there's plenty of
@@ -87,15 +87,23 @@ SECTIONS * Very old versions of ld do not recognize this name token; use the constant. */ #define PT_GNU_EH_FRAME 0x6474e550 +#define PT_GNU_STACK 0x6474e551 +#define PT_GNU_PROPERTY 0x6474e553 /* * We must supply the ELF program headers explicitly to get just one * PT_LOAD segment, and set the flags explicitly to make segments read-only. */ +#define PF_R FLAGS(4) +#define PF_RW FLAGS(6) +#define PF_RX FLAGS(5) + PHDRS { - text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */ - dynamic PT_DYNAMIC FLAGS(4); /* PF_R */ - note PT_NOTE FLAGS(4); /* PF_R */ - eh_frame_hdr PT_GNU_EH_FRAME; + text PT_LOAD PF_RX FILEHDR PHDRS; + dynamic PT_DYNAMIC PF_R; + note PT_NOTE PF_R; + eh_frame_hdr PT_GNU_EH_FRAME PF_R; + gnu_stack PT_GNU_STACK PF_RW; + gnu_property PT_GNU_PROPERTY PF_R; }
Signature · 1 line
2.52.0
Re: [PATCH v4.1 07/10] x86/entry/vdso: include GNU_PROPERTY and GNU_STACK PHDRs
Quoted 12 lines · H. Peter Anvin
Currently the vdso doesn't include .note.gnu.property or a GNU noexec stack annotation (the -z noexecstack in the linker script is ineffective because we specify PHDRs explicitly.)
The motivation is that the dynamic linker currently do not check these.
However, this is a weak excuse: the vdso*.so are also supposed to be usable at link libraries, and there is no reason why the dynamic linker might not want or need to check these in the future, so add them back in -- it is trivial enough.
Quoted 6 lines
Use symbolic constants for the PHDR permission flags.
[ v4: drop unrelated formatting changes ]
[ v4.1: drop one last bogus formatting change (Brian Gerst) ]Signed-off-by H. Peter Anvin (Intel) <[email protected]>
The v4/v4.1 notes will presumably want dropping before committing?
H. Peter Anvin
maintainerPATCH v4 8/10 x86/vdso: abstract out vdso system call internals
In reply to H. Peter AnvinAbstract out the calling of true system calls from the vdso into macros.
It has been a very long time since gcc did not allow %ebx or %ebp in inline asm in 32-bit PIC mode; remove the corresponding hacks.
Remove the use of memory output constraints in gettimeofday.h in favor of "memory" clobbers. The resulting code is identical for the current use cases, as the system call is usually a terminal fallback anyway, and it merely complicates the macroization.
This patch adds only a handful of more lines of code than it removes, and in fact could be made substantially smaller by removing the macros for the argument counts that aren't currently used, however, it seems better to be general from the start.
[ v3: remove stray comment from prototyping; remove VDSO_SYSCALL6()
since it would require special handling on 32 bits and is currently unused. (Uros Bizjak).
Indent nested preprocessor directives. ]
--- arch/x86/include/asm/vdso/gettimeofday.h | 108 ++--------------------- arch/x86/include/asm/vdso/sys_call.h | 103 +++++++++++++++++++++ 2 files changed, 111 insertions(+), 100 deletions(-) create mode 100644 arch/x86/include/asm/vdso/sys_call.h
arch/x86/include/asm/vdso/gettimeofday.h +8 -100
@@ -18,6 +18,7 @@#include <asm/msr.h> #include <asm/pvclock.h> #include <clocksource/hyperv_timer.h> +#include <asm/vdso/sys_call.h> #define VDSO_HAS_TIME 1
@@ -53,130 +54,37 @@ extern struct ms_hyperv_tsc_page hvclock_page __attribute__((visibility("hidden")));#endif -#ifndef BUILD_VDSO32 -static __always_inline long clock_gettime_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) { -long ret; - - asm ("syscall" :"=a" (ret),"=m" (*_ts) : -"0" (__NR_clock_gettime),"D" (_clkid),"S" (_ts) : -"rcx" ,"r11" ); - -return ret; +return VDSO_SYSCALL2 (clock_gettime,64 ,_clkid,_ts); }static __always_inline long gettimeofday_fallback (struct __kernel_old_timeval *_tv,struct timezone *_tz) { -long ret; - - asm("syscall" :"=a" (ret) : -"0" (__NR_gettimeofday),"D" (_tv),"S" (_tz) :"memory" ); - -return ret; + return VDSO_SYSCALL2(gettimeofday,,_tv,_tz); }static __always_inline long clock_getres_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) { -long ret; - - asm ("syscall" :"=a" (ret),"=m" (*_ts) : -"0" (__NR_clock_getres),"D" (_clkid),"S" (_ts) : -"rcx" ,"r11" ); - -return ret; +return VDSO_SYSCALL2 (clock_getres,_time64,_clkid,_ts); } -#else - -static __always_inline -long clock_gettime_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) -{ -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_gettime64), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} +#ifndef CONFIG_X86_64 static __always_inline long clock_gettime32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) { -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_gettime), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} - -static __always_inline -long gettimeofday_fallback (struct __kernel_old_timeval *_tv, -struct timezone *_tz) -{ -long ret; - - asm( -"mov %%ebx, %%edx \n" -"mov %2, %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret) - :"0" (__NR_gettimeofday),"g" (_tv),"c" (_tz) - :"memory" ,"edx" ); - -return ret; + return VDSO_SYSCALL2(clock_gettime,,_clkid,_ts); }static __always_inline long -clock_getres_fallback (clockid_t _clkid,struct __kernel_timespec *_ts) -{ -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_getres_time64), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; -} - -static __always_inline -long clock_getres32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) +clock_getres32_fallback (clockid_t _clkid,struct old_timespec32 *_ts) { -long ret; - - asm ( -"mov %%ebx, %%edx \n" -"mov %[clock], %%ebx \n" -"call __kernel_vsyscall \n" -"mov %%edx, %%ebx \n" - :"=a" (ret),"=m" (*_ts) - :"0" (__NR_clock_getres), [clock]"g" (_clkid),"c" (_ts) - :"edx" ); - -return ret; + return VDSO_SYSCALL2(clock_getres,,_clkid,_ts); }#endif
arch/x86/include/asm/vdso/sys_call.h new file +103 -0
103 lines not on this page · raw message
Signature · 1 line
2.52.0
H. Peter Anvin
maintainerPATCH v4 9/10 x86/cpufeature: replace X86_FEATURE_SYSENTER32 with X86_FEATURE_SYSFAST32
In reply to H. Peter AnvinIn most cases, the use of "fast 32-bit system call" depends either on X86_FEATURE_SEP or X86_FEATURE_SYSENTER32 || X86_FEATURE_SYSCALL32. However, nearly all the logic for both is identical.
Define X86_FEATURE_SYSFAST32 which indicates that *either* SYSENTER32 or SYSCALL32 should be used, for either 32- or 64-bit kernels. This defaults to SYSENTER; use SYSCALL if the SYSCALL32 bit is also set.
As this removes ALL existing uses of X86_FEATURE_SYSENTER32, which is a kernel-only synthetic feature bit, simply remove it and replace it with X86_FEATURE_SYSFAST32.
This leaves an unused alternative for a true 32-bit kernel, but that should really not matter in any way.
The clearing of X86_FEATURE_SYSCALL32 can be removed once the patches for automatically clearing disabled features has been merged.
--- arch/x86/Kconfig.cpufeatures | 8 +++++++ arch/x86/entry/vdso/vdso32/system_call.S | 8 ++----- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/centaur.c | 3 --- arch/x86/kernel/cpu/common.c | 8 +++++++ arch/x86/kernel/cpu/intel.c | 4 +--- arch/x86/kernel/cpu/zhaoxin.c | 4 +--- arch/x86/kernel/fred.c | 2 +- arch/x86/xen/setup.c | 28 +++++++++++++++--------- arch/x86/xen/smp_pv.c | 5 ++--- arch/x86/xen/xen-ops.h | 1 - 11 files changed, 42 insertions(+), 31 deletions(-)
arch/x86/Kconfig.cpufeatures +8 -0
@@ -56,6 +56,10 @@ config X86_REQUIRED_FEATURE_MOVBE def_bool y depends on MATOM +config X86_REQUIRED_FEATURE_SYSFAST32 + def_bool y + depends on X86_64 && !X86_FRED + config X86_REQUIRED_FEATURE_CPUID def_bool y depends on X86_64
@@ -120,6 +124,10 @@ config X86_DISABLED_FEATURE_CENTAUR_MCR def_bool y depends on X86_64 +config X86_DISABLED_FEATURE_SYSCALL32 + def_bool y + depends on !X86_64 + config X86_DISABLED_FEATURE_PCID def_bool y depends on !X86_64
arch/x86/entry/vdso/vdso32/system_call.S +2 -6
@@ -52,13 +52,9 @@ __kernel_vsyscall: #define SYSENTER_SEQUENCE "movl %esp, %ebp; sysenter" #define SYSCALL_SEQUENCE "movl %ecx, %ebp; syscall" -#ifdef BUILD_VDSO32_64 /* If SYSENTER (Intel) or SYSCALL32 (AMD) is available, use it. */ - ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSENTER32, \ - SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 -#else - ALTERNATIVE "", SYSENTER_SEQUENCE, X86_FEATURE_SEP -#endif + ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSFAST32, \ + SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 /* Enter using int $0x80 */ int $0x80
arch/x86/include/asm/cpufeatures.h +1 -1
@@ -84,7 +84,7 @@#define X86_FEATURE_PEBS ( 3*32+12)/* "pebs" Precise-Event Based Sampling */ #define X86_FEATURE_BTS ( 3*32+13)/* "bts" Branch Trace Store */ #define X86_FEATURE_SYSCALL32 ( 3*32+14)/* syscall in IA32 userspace */ -#define X86_FEATURE_SYSENTER32 ( 3*32+15)/* sysenter in IA32 userspace */ +#define X86_FEATURE_SYSFAST32 ( 3*32+15)/* sysenter/syscall in IA32 userspace */ #define X86_FEATURE_REP_GOOD ( 3*32+16)/* "rep_good" REP microcode works well */ #define X86_FEATURE_AMD_LBR_V2 ( 3*32+17)/* "amd_lbr_v2" AMD Last Branch Record Extension Version 2 */ #define X86_FEATURE_CLEAR_CPU_BUF ( 3*32+18)/* Clear CPU buffers using VERW */
arch/x86/kernel/cpu/centaur.c +0 -3
@@ -102,9 +102,6 @@ static void early_init_centaur(struct cpuinfo_x86 *c) (c->x86 >= 7))set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC ); -#ifdef CONFIG_X86_64 -set_cpu_cap (c,X86_FEATURE_SYSENTER32 ); -#endif if (c->x86_power & (1 <<8 )) {set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC );set_cpu_cap (c,X86_FEATURE_NONSTOP_TSC );
arch/x86/kernel/cpu/common.c +8 -0
@@ -1068,6 +1068,9 @@ void get_cpu_cap(struct cpuinfo_x86 *c)init_scattered_cpuid_features (c);init_speculation_control (c); +if (IS_ENABLED (CONFIG_X86_64 ) ||cpu_has (c,X86_FEATURE_SEP )) +set_cpu_cap (c,X86_FEATURE_SYSFAST32 ); +/* * Clear/Set all flags overridden by options, after probe. * This needs to happen each time we re-probe, which may happen
@@ -1813,6 +1816,11 @@ static void __init early_identify_cpu(struct cpuinfo_x86 *c)* that it can't be enabled in 32-bit mode. */ setup_clear_cpu_cap (X86_FEATURE_PCID ); + +/* +* Never use SYSCALL on a 32-bit kernel +*/ +setup_clear_cpu_cap (X86_FEATURE_SYSCALL32 );#endif /*
arch/x86/kernel/cpu/intel.c +1 -3
@@ -236,9 +236,7 @@ static void early_init_intel(struct cpuinfo_x86 *c)clear_cpu_cap (c,X86_FEATURE_PSE ); } -#ifdef CONFIG_X86_64 -set_cpu_cap (c,X86_FEATURE_SYSENTER32 ); -#else +#ifndef CONFIG_X86_64 /* Netburst reports 64 bytes clflush size, but does IO in 128 bytes */ if (c->x86 ==15 && c->x86_cache_alignment ==64 ) c->x86_cache_alignment =128 ;
arch/x86/kernel/cpu/zhaoxin.c +1 -3
@@ -59,9 +59,7 @@ static void early_init_zhaoxin(struct cpuinfo_x86 *c) {if (c->x86 >=0x6 )set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC ); -#ifdef CONFIG_X86_64 -set_cpu_cap (c,X86_FEATURE_SYSENTER32 ); -#endif +if (c->x86_power & (1 <<8 )) {set_cpu_cap (c,X86_FEATURE_CONSTANT_TSC );set_cpu_cap (c,X86_FEATURE_NONSTOP_TSC );
arch/x86/kernel/fred.c +1 -1
@@ -68,7 +68,7 @@ void cpu_init_fred_exceptions(void)idt_invalidate ();/* Use int $0x80 for 32-bit system calls in FRED mode */ -setup_clear_cpu_cap (X86_FEATURE_SYSENTER32 ); +setup_clear_cpu_cap (X86_FEATURE_SYSFAST32 );setup_clear_cpu_cap (X86_FEATURE_SYSCALL32 ); }
arch/x86/xen/setup.c +18 -10
@@ -990,13 +990,6 @@ static int register_callback(unsigned type, const void *func)return HYPERVISOR_callback_op (CALLBACKOP_register, &callback); } -void xen_enable_sysenter (void ) -{ -if (cpu_feature_enabled (X86_FEATURE_SYSENTER32 ) && -register_callback (CALLBACKTYPE_sysenter, xen_entry_SYSENTER_compat)) -setup_clear_cpu_cap (X86_FEATURE_SYSENTER32 ); -} -void xen_enable_syscall (void ) {int ret;
@@ -1008,11 +1001,27 @@ void xen_enable_syscall(void)mechanism for syscalls. */ } -if (cpu_feature_enabled (X86_FEATURE_SYSCALL32 ) && -register_callback (CALLBACKTYPE_syscall32, xen_entry_SYSCALL_compat)) +if (!cpu_feature_enabled (X86_FEATURE_SYSFAST32 )) +return ; + +if (cpu_feature_enabled (X86_FEATURE_SYSCALL32 )) { +/* Use SYSCALL32 */ + ret =register_callback (CALLBACKTYPE_syscall32, + xen_entry_SYSCALL_compat); + + }else { +/* Use SYSENTER32 */ + ret =register_callback (CALLBACKTYPE_sysenter, + xen_entry_SYSENTER_compat); + } + +if (ret) {setup_clear_cpu_cap (X86_FEATURE_SYSCALL32 ); +setup_clear_cpu_cap (X86_FEATURE_SYSFAST32 ); + } } +static void __init xen_pvmmu_arch_setup (void ) {HYPERVISOR_vm_assist (VMASST_CMD_enable, VMASST_TYPE_writable_pagetables);
@@ -1022,7 +1031,6 @@ static void __init xen_pvmmu_arch_setup(void) register_callback(CALLBACKTYPE_failsafe, xen_failsafe_callback))BUG (); -xen_enable_sysenter ();xen_enable_syscall (); }
arch/x86/xen/smp_pv.c +2 -3
@@ -65,10 +65,9 @@ static void cpu_bringup(void)touch_softlockup_watchdog ();/* PVH runs in ring 0 and allows us to do native syscalls. Yay! */ -if (!xen_feature (XENFEAT_supervisor_mode_kernel)) { -xen_enable_sysenter (); +if (!xen_feature (XENFEAT_supervisor_mode_kernel))xen_enable_syscall (); - } + cpu =smp_processor_id ();identify_secondary_cpu (cpu);set_cpu_sibling_map (cpu);
arch/x86/xen/xen-ops.h +0 -1
@@ -60,7 +60,6 @@ phys_addr_t __init xen_find_free_area(phys_addr_t size);char *__init xen_memory_setup (void );void __init xen_arch_setup (void );void xen_banner (void ); -void xen_enable_sysenter (void );void xen_enable_syscall (void );void xen_vcpu_restore (void );
Signature · 1 line
2.52.0
H. Peter Anvin
maintainerPATCH v4 10/10 x86/entry/vdso32: when using int $0x80, use it directly
In reply to H. Peter AnvinWhen neither sysenter32 nor syscall32 is available (on either FRED-capable 64-bit hardware or old 32-bit hardware), there is no reason to do a bunch of stack shuffling in __kernel_vsyscall. Unfortunately, just overwriting the initial "push" instructions will mess up the CFI annotations, so suffer the 3-byte NOP if not applicable.
Similarly, inline the int $0x80 when doing inline system calls in the vdso instead of calling __kernel_vsyscall.
--- arch/x86/entry/vdso/vdso32/system_call.S | 18 ++++++++++++++---- arch/x86/include/asm/vdso/sys_call.h | 4 +++- 2 files changed, 17 insertions(+), 5 deletions(-)
arch/x86/entry/vdso/vdso32/system_call.S +14 -4
@@ -14,6 +14,18 @@ ALIGN __kernel_vsyscall: CFI_STARTPROC + + /* + * If using int $0x80, there is no reason to muck about with the + * stack here. Unfortunately just overwriting the push instructions + * would mess up the CFI annotations, but it is only a 3-byte + * NOP in that case. This could be avoided by patching the + * vdso symbol table (not the code) and entry point, but that + * would a fair bit of tooling work or by simply compiling + * two different vDSO images, but that doesn't seem worth it. + */ + ALTERNATIVE "int $0x80; ret", "", X86_FEATURE_SYSFAST32 + /* * Reshuffle regs so that all of any of the entry instructions * will preserve enough state.
@@ -52,11 +64,9 @@ __kernel_vsyscall: #define SYSENTER_SEQUENCE "movl %esp, %ebp; sysenter" #define SYSCALL_SEQUENCE "movl %ecx, %ebp; syscall" - /* If SYSENTER (Intel) or SYSCALL32 (AMD) is available, use it. */ - ALTERNATIVE_2 "", SYSENTER_SEQUENCE, X86_FEATURE_SYSFAST32, \ - SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 + ALTERNATIVE SYSENTER_SEQUENCE, SYSCALL_SEQUENCE, X86_FEATURE_SYSCALL32 - /* Enter using int $0x80 */ + /* Re-enter using int $0x80 */ int $0x80 SYM_INNER_LABEL(int80_landing_pad, SYM_L_GLOBAL)
arch/x86/include/asm/vdso/sys_call.h +3 -1
@@ -20,7 +20,9 @@# define __sys_reg4 "r10"# define __sys_reg5 "r8"#else -# define __sys_instr "call __kernel_vsyscall" +# define __sys_instr ALTERNATIVE("ds;ds;ds;int $0x80", \ + "call __kernel_vsyscall", \ + X86_FEATURE_SYSFAST32)# define __sys_clobber "memory"# define __sys_nr (x,y) __NR_ ## x ## y# define __sys_reg1 "ebx"
Signature · 1 line
2.52.0
[stripped down the list of recipients quite a bit]
Quoted 6 lines · H. Peter Anvin
[...]
This patchset cleans up the vdso build by building the 32- and 64-bit vdsos in separate directories, moving the vdso2c tool toarch/x86/tools, and by merging common code and especiallyMakefilerules between the 32- and 64-bit vdsos to the greatest extent possible. [...]
Lo! My daily -next builds for Fedora failed on x86_64 (other archs worked fine). Haven't checked, but from the error message I wonder if this might be due to the changes from this patch-set that showed up in -next today:
+ /usr/bin/make -s 'HOSTCFLAGS=-O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 ' 'HOSTLDFLAGS=-Wl,-z,relro -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-hardened-ld-errors -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes ' ARCH=x86_64 INSTALL_MOD_PATH=/builddir/build/BUILD/kernel-6.19.0-build/BUILDROOT vdso_install KERNELRELEASE=6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64+rt
readelf: Error: 'arch/x86/entry/vdso/vdso32.so.dbg': No such file
readelf: Error: 'arch/x86/entry/vdso/vdso64.so.dbg': No such filescripts/Makefile.vdsoinst:33: warning: overriding recipe for target '/builddir/build/BUILD/kernel-6.19.0-build/BUILDROOT/lib/modules/6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64+rt/vdso/.build-id/.debug' scripts/Makefile.vdsoinst:33: warning: ignoring old recipe for target '/builddir/build/BUILD/kernel-6.19.0-build/BUILDROOT/lib/modules/6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64+rt/vdso/.build-id/.debug' make[2]: *** No rule to make target 'arch/x86/entry/vdso/vdso32.so.dbg', needed by '/builddir/build/BUILD/kernel-6.19.0-build/BUILDROOT/lib/modules/6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64+rt/vdso/vdso32.so'. Stop. make[1]: *** [/builddir/build/BUILD/kernel-6.19.0-build/kernel-next-20260115/linux-6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64/Makefile:1459: vdso_install] Error 2
make: *** [Makefile:256: __sub-make] Error 2
Ciao, Thorsten
H. Peter Anvin
maintainerQuoted 24 lines · Thorsten Leemhuis
[stripped down the list of recipients quite a bit]
Quoted 6 lines · H. Peter Anvin
[...]
This patchset cleans up the vdso build by building the 32- and 64-bit vdsos in separate directories, moving the vdso2c tool toarch/x86/tools, and by merging common code and especiallyMakefilerules between the 32- and 64-bit vdsos to the greatest extent possible. [...]Lo! My daily -next builds for Fedora failed on
x86_64(other archs worked fine). Haven't checked, but from the error message I wonder if this might be due to the changes from this patch-set that showed up in -next today:+
/usr/bin/make-s 'HOSTCFLAGS=-O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 ' 'HOSTLDFLAGS=-Wl,-z,relro -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-hardened-ld-errors -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes ' ARCH=x86_64 INSTALL_MOD_PATH=/builddir/build/BUILD/kernel-6.19.0-build/BUILDROOTvdso_installKERNELRELEASE=6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64+rt
readelf: Error: 'arch/x86/entry/vdso/vdso32.so.dbg': No such file
readelf: Error: 'arch/x86/entry/vdso/vdso64.so.dbg': No such filescripts/Makefile.vdsoinst:33: warning: overriding recipe for target '/builddir/build/BUILD/kernel-6.19.0-build/BUILDROOT/lib/modules/6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64+rt/vdso/.build-id/.debug'scripts/Makefile.vdsoinst:33: warning: ignoring old recipe for target '/builddir/build/BUILD/kernel-6.19.0-build/BUILDROOT/lib/modules/6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64+rt/vdso/.build-id/.debug' make[2]: *** No rule to make target 'arch/x86/entry/vdso/vdso32.so.dbg', needed by '/builddir/build/BUILD/kernel-6.19.0-build/BUILDROOT/lib/modules/6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64+rt/vdso/vdso32.so'. Stop. make[1]: *** [/builddir/build/BUILD/kernel-6.19.0-build/kernel-next-20260115/linux-6.19.0-0.0.next.20260115.439.vanilla.fc44.x86_64/Makefile:1459:vdso_install] Error 2
make: *** [Makefile:256: __sub-make] Error 2
Quoted 4 lines
Ciao, Thorsten
Looks like it. Specifically it looks like it needs a tweak to make vdso_install. I'll look at it in a few hours.