x86: Rewrite 64-bit syscall code
Subsystems (5)
Andy Lutomirski
maintainerv2 cover x86: Rewrite 64-bit syscall code
This is kind of like the 32-bit and compat code, except that I preserved the fast path this time. I was unable to measure any significant performance change on my laptop in the fast path.
Changes from v1:
- Various tidying up.
- Remove duplicate tables (folded in, so the fastpath table isn't in this set).
- Rebased to 4.5-rc1
- Remove enter_from_user_mode stuff -- let's get the basics in first.
Andy Lutomirski (10):
selftests/x86: Extend Makefile to allow 64-bit-only tests selftests/x86: Add check_initial_reg_state
x86/syscalls: Refactor syscalltbl.sh
x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32 x86/syscalls: Move compat syscall entry handling into syscalltbl.sh x86/syscalls: Add syscall entry qualifiers
x86/entry/64: Always run ptregs-using syscalls on the slow path x86/entry/64: Call all native slow-path syscalls with full pt-regs x86/entry/64: Stop using int_ret_from_sys_call in ret_from_fork x86/entry/64: Migrate the 64-bit syscall slow path to C
arch/x86/entry/common.c | 26 ++ arch/x86/entry/entry_64.S | 271 +++++++-------------- arch/x86/entry/syscall_32.c | 10 +- arch/x86/entry/syscall_64.c | 13 +- arch/x86/entry/syscalls/syscall_64.tbl | 18 +- arch/x86/entry/syscalls/syscalltbl.sh | 58 ++++- arch/x86/kernel/asm-offsets_32.c | 2 +- arch/x86/kernel/asm-offsets_64.c | 10 +- arch/x86/um/sys_call_table_32.c | 4 +- arch/x86/um/sys_call_table_64.c | 7 +- arch/x86/um/user-offsets.c | 6 +- tools/testing/selftests/x86/Makefile | 14 +- .../selftests/x86/check_initial_reg_state.c | 109 +++++++++ 13 files changed, 317 insertions(+), 231 deletions(-) create mode 100644 tools/testing/selftests/x86/check_initial_reg_state.c
Signature · 1 line
2.5.0
Andy Lutomirski
maintainerPATCH v2 1/10 selftests/x86: Extend Makefile to allow 64-bit-only tests
Previously the Makefile supported 32-bit-only tests and tests that were 32-bit and 64-bit. This adds the support for tests that are only built as 64-bit binaries.
There aren't any yet, but there might be a few some day.
Cc Shuah Khan <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
--- tools/testing/selftests/x86/Makefile | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tools/testing/selftests/x86/Makefile +3 -2
@@ -11,8 +11,9 @@ TARGETS_C_32BIT_ONLY := entry_from_vm86 syscall_arg_fault sigreturn test_syscall vdso_restorer TARGETS_C_32BIT_ALL := $(TARGETS_C_BOTHBITS) $(TARGETS_C_32BIT_ONLY) +TARGETS_C_64BIT_ALL := $(TARGETS_C_BOTHBITS) $(TARGETS_C_64BIT_ONLY) BINARIES_32 := $(TARGETS_C_32BIT_ALL:%=%_32) -BINARIES_64 := $(TARGETS_C_BOTHBITS:%=%_64) +BINARIES_64 := $(TARGETS_C_64BIT_ALL:%=%_64) CFLAGS := -O2 -g -std=gnu99 -pthread -Wall
@@ -40,7 +41,7 @@ clean: $(TARGETS_C_32BIT_ALL:%=%_32): %_32: %.c $(CC) -m32 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $^ -lrt -ldl -lm -$(TARGETS_C_BOTHBITS:%=%_64): %_64: %.c +$(TARGETS_C_64BIT_ALL:%=%_64): %_64: %.c $(CC) -m64 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $^ -lrt -ldl # x86_64 users should be encouraged to install 32-bit libraries
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] selftests/x86: Extend Makefile to allow 64-bit-only tests
selftests/x86: Extend Makefile to allow 64-bit-only tests
Commit-ID: c31b34255b48d1a169693c9c70c49ad6418cfd20 Gitweb: http://git.kernel.org/tip/c31b34255b48d1a169693c9c70c49ad6418cfd20 Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:19 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:36 +0100
selftests/x86: Extend Makefile to allow 64-bit-only tests
Previously the Makefile supported 32-bit-only tests and tests that were 32-bit and 64-bit. This adds the support for tests that are only built as 64-bit binaries.
There aren't any yet, but there might be a few some day.
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Shuah Khan <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/99789bfe65706e6df32cc7e13f656e8c9fa92031.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- tools/testing/selftests/x86/Makefile | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tools/testing/selftests/x86/Makefile +3 -2
@@ -11,8 +11,9 @@ TARGETS_C_32BIT_ONLY := entry_from_vm86 syscall_arg_fault sigreturn test_syscall vdso_restorer TARGETS_C_32BIT_ALL := $(TARGETS_C_BOTHBITS) $(TARGETS_C_32BIT_ONLY) +TARGETS_C_64BIT_ALL := $(TARGETS_C_BOTHBITS) $(TARGETS_C_64BIT_ONLY) BINARIES_32 := $(TARGETS_C_32BIT_ALL:%=%_32) -BINARIES_64 := $(TARGETS_C_BOTHBITS:%=%_64) +BINARIES_64 := $(TARGETS_C_64BIT_ALL:%=%_64) CFLAGS := -O2 -g -std=gnu99 -pthread -Wall
@@ -40,7 +41,7 @@ clean: $(TARGETS_C_32BIT_ALL:%=%_32): %_32: %.c $(CC) -m32 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $^ -lrt -ldl -lm -$(TARGETS_C_BOTHBITS:%=%_64): %_64: %.c +$(TARGETS_C_64BIT_ALL:%=%_64): %_64: %.c $(CC) -m64 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $^ -lrt -ldl # x86_64 users should be encouraged to install 32-bit libraries
Andy Lutomirski
maintainerPATCH v2 2/10 selftests/x86: Add check_initial_reg_state
In reply to Andy LutomirskiThis checks that ELF binaries are started with an appropriately blank register state.
(There's currently a nasty special case in the entry asm to arrange
for this. I'm planning on removing the special case, and this will help make sure I don't break it.)
Signed-off-by Andy Lutomirski <[email protected]>
--- tools/testing/selftests/x86/Makefile | 9 +- .../selftests/x86/check_initial_reg_state.c | 109 +++++++++++++++++++++ 2 files changed, 117 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/x86/check_initial_reg_state.c
tools/testing/selftests/x86/Makefile +8 -1
@@ -4,7 +4,8 @@ include ../lib.mk .PHONY: all all_32 all_64 warn_32bit_failure clean -TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt ptrace_syscall +TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt ptrace_syscall \ + check_initial_reg_state TARGETS_C_32BIT_ONLY := entry_from_vm86 syscall_arg_fault sigreturn test_syscall_vdso unwind_vdso \ test_FCMOV test_FCOMI test_FISTTP \ ldt_gdt \
@@ -66,3 +67,9 @@ endif sysret_ss_attrs_64: thunks.S ptrace_syscall_32: raw_syscall_helper_32.S test_syscall_vdso_32: thunks_32.S + +# check_initial_reg_state is special: it needs a custom entry, and it +# needs to be static so that its interpreter doesn't destroy its initial +# state. +check_initial_reg_state_32: CFLAGS += -Wl,-ereal_start -static +check_initial_reg_state_64: CFLAGS += -Wl,-ereal_start -static
tools/testing/selftests/x86/check_initial_reg_state.c new file +109 -0
109 lines not on this page · raw message
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] selftests/x86: Add check_initial_reg_state()
selftests/x86: Add check_initial_reg_state()
Commit-ID: e21d50f3864e2a8995f5d2a41dea3f0fa07758b4 Gitweb: http://git.kernel.org/tip/e21d50f3864e2a8995f5d2a41dea3f0fa07758b4 Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:20 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:37 +0100
selftests/x86: Add check_initial_reg_state()
This checks that ELF binaries are started with an appropriately blank register state.
( There's currently a nasty special case in the entry asm to
arrange for this. I'm planning on removing the special case, and this will help make sure I don't break it. )
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/ef54f8d066b30a3eb36bbf26300eebb242185700.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- tools/testing/selftests/x86/Makefile | 9 +- .../selftests/x86/check_initial_reg_state.c | 109 +++++++++++++++++++++ 2 files changed, 117 insertions(+), 1 deletion(-)
tools/testing/selftests/x86/Makefile +8 -1
@@ -4,7 +4,8 @@ include ../lib.mk .PHONY: all all_32 all_64 warn_32bit_failure clean -TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt ptrace_syscall +TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt ptrace_syscall \ + check_initial_reg_state TARGETS_C_32BIT_ONLY := entry_from_vm86 syscall_arg_fault sigreturn test_syscall_vdso unwind_vdso \ test_FCMOV test_FCOMI test_FISTTP \ ldt_gdt \
@@ -66,3 +67,9 @@ endif sysret_ss_attrs_64: thunks.S ptrace_syscall_32: raw_syscall_helper_32.S test_syscall_vdso_32: thunks_32.S + +# check_initial_reg_state is special: it needs a custom entry, and it +# needs to be static so that its interpreter doesn't destroy its initial +# state. +check_initial_reg_state_32: CFLAGS += -Wl,-ereal_start -static +check_initial_reg_state_64: CFLAGS += -Wl,-ereal_start -static
tools/testing/selftests/x86/check_initial_reg_state.c new file +109 -0
109 lines not on this page · raw message
Andy Lutomirski
maintainerPATCH v2 3/10 x86/syscalls: Refactor syscalltbl.sh
In reply to Andy LutomirskiThis splits out the code to emit a syscall line.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/entry/syscalls/syscalltbl.sh | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)
arch/x86/entry/syscalls/syscalltbl.sh +13 -5
@@ -3,13 +3,21 @@ in="$1" out="$2" +emit() { + abi="$1" + nr="$2" + entry="$3" + compat="$4" + if [ -n "$compat" ]; then + echo "__SYSCALL_${abi}($nr, $entry, $compat)" + elif [ -n "$entry" ]; then + echo "__SYSCALL_${abi}($nr, $entry, $entry)" + fi +} + grep '^[0-9]' "$in" | sort -n | ( while read nr abi name entry compat; do abi=`echo "$abi" | tr '[a-z]' '[A-Z]'` - if [ -n "$compat" ]; then - echo "__SYSCALL_${abi}($nr, $entry, $compat)" - elif [ -n "$entry" ]; then - echo "__SYSCALL_${abi}($nr, $entry, $entry)" - fi + emit "$abi" "$nr" "$entry" "$compat" done ) > "$out"
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] x86/syscalls: Refactor syscalltbl.sh
x86/syscalls: Refactor syscalltbl.sh
Commit-ID: fba324744bfd2a7948a7710d7a021d76dafb9b67 Gitweb: http://git.kernel.org/tip/fba324744bfd2a7948a7710d7a021d76dafb9b67 Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:21 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:37 +0100
x86/syscalls: Refactor syscalltbl.sh
This splits out the code to emit a syscall line.
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/1bfcbba991f5cfaa9291ff950a593daa972a205f.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/entry/syscalls/syscalltbl.sh | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)
arch/x86/entry/syscalls/syscalltbl.sh +13 -5
@@ -3,13 +3,21 @@ in="$1" out="$2" +emit() { + abi="$1" + nr="$2" + entry="$3" + compat="$4" + if [ -n "$compat" ]; then + echo "__SYSCALL_${abi}($nr, $entry, $compat)" + elif [ -n "$entry" ]; then + echo "__SYSCALL_${abi}($nr, $entry, $entry)" + fi +} + grep '^[0-9]' "$in" | sort -n | ( while read nr abi name entry compat; do abi=`echo "$abi" | tr '[a-z]' '[A-Z]'` - if [ -n "$compat" ]; then - echo "__SYSCALL_${abi}($nr, $entry, $compat)" - elif [ -n "$entry" ]; then - echo "__SYSCALL_${abi}($nr, $entry, $entry)" - fi + emit "$abi" "$nr" "$entry" "$compat" done ) > "$out"
Andy Lutomirski
maintainerPATCH v2 4/10 x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
In reply to Andy LutomirskiThe common/64/x32 distinction has no effect other than determining which kernels actually support the syscall. Move the logic into syscalltbl.sh.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/entry/syscall_64.c | 8 -------- arch/x86/entry/syscalls/syscalltbl.sh | 17 ++++++++++++++++- arch/x86/kernel/asm-offsets_64.c | 6 ------ arch/x86/um/sys_call_table_64.c | 3 --- arch/x86/um/user-offsets.c | 2 -- 5 files changed, 16 insertions(+), 20 deletions(-)
arch/x86/entry/syscall_64.c +0 -8
@@ -6,14 +6,6 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) - -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64
arch/x86/entry/syscalls/syscalltbl.sh +16 -1
@@ -18,6 +18,21 @@ emit() { grep '^[0-9]' "$in" | sort -n | ( while read nr abi name entry compat; do abi=`echo "$abi" | tr '[a-z]' '[A-Z]'` - emit "$abi" "$nr" "$entry" "$compat" + if [ "$abi" == "COMMON" -o "$abi" == "64" ]; then + # COMMON is the same as 64, except that we don't expect X32 + # programs to use it. Our expectation has nothing to do with + # any generated code, so treat them the same. + emit 64 "$nr" "$entry" "$compat" + elif [ "$abi" == "X32" ]; then + # X32 is equivalent to 64 on an X32-compatible kernel. + echo "#ifdef CONFIG_X86_X32_ABI" + emit 64 "$nr" "$entry" "$compat" + echo "#endif" + elif [ "$abi" == "I386" ]; then + emit "$abi" "$nr" "$entry" "$compat" + else + echo "Unknown abi $abi" >&2 + exit 1 + fi done ) > "$out"
arch/x86/kernel/asm-offsets_64.c +0 -6
@@ -5,12 +5,6 @@#include <asm/ia32.h> #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) [nr] = 1, -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif static char syscalls_64[] = {#include <asm/syscalls_64.h> };
arch/x86/um/sys_call_table_64.c +0 -3
@@ -35,9 +35,6 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h>
arch/x86/um/user-offsets.c +0 -2
@@ -15,8 +15,6 @@ static char syscalls[] = { };#else #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ static char syscalls[] = {#include <asm/syscalls_64.h> };
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
Commit-ID: 32324ce15ea8cb4c8acc28acb2fd36fabf73e9db Gitweb: http://git.kernel.org/tip/32324ce15ea8cb4c8acc28acb2fd36fabf73e9db Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:22 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:37 +0100
x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
The common/64/x32 distinction has no effect other than determining which kernels actually support the syscall. Move the logic into syscalltbl.sh.
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/58d4a95f40e43b894f93288b4a3633963d0ee22e.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/entry/syscall_64.c | 8 -------- arch/x86/entry/syscalls/syscalltbl.sh | 17 ++++++++++++++++- arch/x86/kernel/asm-offsets_64.c | 6 ------ arch/x86/um/sys_call_table_64.c | 3 --- arch/x86/um/user-offsets.c | 2 -- 5 files changed, 16 insertions(+), 20 deletions(-)
arch/x86/entry/syscall_64.c +0 -8
@@ -6,14 +6,6 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) - -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64
arch/x86/entry/syscalls/syscalltbl.sh +16 -1
@@ -18,6 +18,21 @@ emit() { grep '^[0-9]' "$in" | sort -n | ( while read nr abi name entry compat; do abi=`echo "$abi" | tr '[a-z]' '[A-Z]'` - emit "$abi" "$nr" "$entry" "$compat" + if [ "$abi" == "COMMON" -o "$abi" == "64" ]; then + # COMMON is the same as 64, except that we don't expect X32 + # programs to use it. Our expectation has nothing to do with + # any generated code, so treat them the same. + emit 64 "$nr" "$entry" "$compat" + elif [ "$abi" == "X32" ]; then + # X32 is equivalent to 64 on an X32-compatible kernel. + echo "#ifdef CONFIG_X86_X32_ABI" + emit 64 "$nr" "$entry" "$compat" + echo "#endif" + elif [ "$abi" == "I386" ]; then + emit "$abi" "$nr" "$entry" "$compat" + else + echo "Unknown abi $abi" >&2 + exit 1 + fi done ) > "$out"
arch/x86/kernel/asm-offsets_64.c +0 -6
@@ -5,12 +5,6 @@#include <asm/ia32.h> #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) [nr] = 1, -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif static char syscalls_64[] = {#include <asm/syscalls_64.h> };
arch/x86/um/sys_call_table_64.c +0 -3
@@ -35,9 +35,6 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h>
arch/x86/um/user-offsets.c +0 -2
@@ -15,8 +15,6 @@ static char syscalls[] = { };#else #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ static char syscalls[] = {#include <asm/syscalls_64.h> };
H. Peter Anvin
reviewerRe: [tip:x86/asm] x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
Quoted 126 lines · tip-bot for Andy Lutomirski
Commit-ID:
32324ce15ea8cb4c8acc28acb2fd36fabf73e9dbGitweb:
http://git.kernel.org/tip/32324ce15ea8cb4c8acc28acb2fd36fabf73e9dbAuthor: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:22 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:37 +0100x86/syscalls: Remove
__SYSCALL_COMMONand__SYSCALL_X32The common/64/x32 distinction has no effect other than determining which kernels actually support the syscall. Move the logic into
syscalltbl.sh.Signed-off-by Andy Lutomirski <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Borislav Petkov <[email protected]>Cc Brian Gerst <[email protected]>Cc Denys Vlasenko <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc H. Peter Anvin <[email protected]>Cc Linus Torvalds <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Thomas Gleixner <[email protected]>Link:
http://lkml.kernel.org/r/58d4a95f40e43b894f93288b4a3633963d0ee22e.1454022279.git.luto@kernel.orgSigned-off-by Ingo Molnar <[email protected]>---
arch/x86/entry/syscall_64.c | 8 -------- arch/x86/entry/syscalls/syscalltbl.sh | 17 ++++++++++++++++- arch/x86/kernel/asm-offsets_64.c | 6 ------ arch/x86/um/sys_call_table_64.c | 3 --- arch/x86/um/user-offsets.c | 2 -- 5 files changed, 16 insertions(+), 20 deletions(-)arch/x86/entry/syscall_64.c +0
-8@@ -6,14 +6,6 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) - -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long,unsigned long, unsigned long) ;
#include <asm/syscalls_64.h>
#undef__SYSCALL_64arch/x86/entry/syscalls/syscalltbl.sh +0
-0@@ -18,6 +18,21 @@ emit() {grep '^[0-9]' "$in" | sort -n | (
while read nr abi name entry compat; do
abi=echo "$abi" | tr '[a-z]' '[A-Z]'arch/x86/entry/syscalls/syscalltbl.sh +16
-1- emit "$abi" "$nr" "$entry" "$compat" + if [ "$abi" == "COMMON" -o "$abi" == "64" ]; then + # COMMON is the same as 64, except that we don't expect X32 + # programs to use it. Our expectation has nothing to do with + # any generated code, so treat them the same. + emit 64 "$nr" "$entry" "$compat" + elif [ "$abi" == "X32" ]; then + # X32 is equivalent to 64 on an X32-compatible kernel. + echo "#ifdef CONFIG_X86_X32_ABI" + emit 64 "$nr" "$entry" "$compat" + echo "#endif" + elif [ "$abi" == "I386" ]; then + emit "$abi" "$nr" "$entry" "$compat" + else + echo "Unknown abi $abi" >&2 + exit 1 + fi done) > "$out"
arch/x86/kernel/asm-offsets_64.c +0
-6@@ -5,12 +5,6 @@#include <asm/ia32.h> #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) [nr] = 1, -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif static char syscalls_64[] = {#include <asm/syscalls_64.h> };arch/x86/um/sys_call_table_64.c +0
-3@@ -35,9 +35,6 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long,unsigned long, unsigned long) ;
#include <asm/syscalls_64.h>arch/x86/um/user-offsets.c +0
-0
arch/x86/um/user-offsets.c static char syscalls[] = { · line 20
Quoted 8 lines
arch/x86/um/user-offsets.c +0
-2@@ -15,8 +15,6 @@ static char syscalls[] = { };#else #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ static char syscalls[] = {#include <asm/syscalls_64.h> };
I am really unhappy about this change. syscalltbl.sh is written so that other architectures can use it, and so I would really prefer for it to stay arch-neutral and encourage other arches to use it, too.
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.
Re: [tip:x86/asm] x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
Quoted 122 lines · H. Peter Anvin
Quoted 121 lines · tip-bot for Andy Lutomirski
Commit-ID:
32324ce15ea8cb4c8acc28acb2fd36fabf73e9dbGitweb:
http://git.kernel.org/tip/32324ce15ea8cb4c8acc28acb2fd36fabf73e9dbAuthor: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:22 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:37 +0100x86/syscalls: Remove
__SYSCALL_COMMONand__SYSCALL_X32The common/64/x32 distinction has no effect other than determining which kernels actually support the syscall. Move the logic into
syscalltbl.sh.Signed-off-by Andy Lutomirski <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Borislav Petkov <[email protected]>Cc Brian Gerst <[email protected]>Cc Denys Vlasenko <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc H. Peter Anvin <[email protected]>Cc Linus Torvalds <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Thomas Gleixner <[email protected]>Link:
http://lkml.kernel.org/r/58d4a95f40e43b894f93288b4a3633963d0ee22e.1454022279.git.luto@kernel.orgSigned-off-by Ingo Molnar <[email protected]>---
arch/x86/entry/syscall_64.c | 8 -------- arch/x86/entry/syscalls/syscalltbl.sh | 17 ++++++++++++++++- arch/x86/kernel/asm-offsets_64.c | 6 ------ arch/x86/um/sys_call_table_64.c | 3 --- arch/x86/um/user-offsets.c | 2 -- 5 files changed, 16 insertions(+), 20 deletions(-)arch/x86/entry/syscall_64.c +0
-8@@ -6,14 +6,6 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) - -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long,unsigned long, unsigned long) ;
#include <asm/syscalls_64.h>
#undef__SYSCALL_64arch/x86/entry/syscalls/syscalltbl.sh +0
-0@@ -18,6 +18,21 @@ emit() {grep '^[0-9]' "$in" | sort -n | (
while read nr abi name entry compat; do
abi=echo "$abi" | tr '[a-z]' '[A-Z]'arch/x86/entry/syscalls/syscalltbl.sh +16
-1- emit "$abi" "$nr" "$entry" "$compat" + if [ "$abi" == "COMMON" -o "$abi" == "64" ]; then + # COMMON is the same as 64, except that we don't expect X32 + # programs to use it. Our expectation has nothing to do with + # any generated code, so treat them the same. + emit 64 "$nr" "$entry" "$compat" + elif [ "$abi" == "X32" ]; then + # X32 is equivalent to 64 on an X32-compatible kernel. + echo "#ifdef CONFIG_X86_X32_ABI" + emit 64 "$nr" "$entry" "$compat" + echo "#endif" + elif [ "$abi" == "I386" ]; then + emit "$abi" "$nr" "$entry" "$compat" + else + echo "Unknown abi $abi" >&2 + exit 1 + fi done) > "$out"
arch/x86/kernel/asm-offsets_64.c +0
-6@@ -5,12 +5,6 @@#include <asm/ia32.h> #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) [nr] = 1, -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif static char syscalls_64[] = {#include <asm/syscalls_64.h> };arch/x86/um/sys_call_table_64.c +0
-3@@ -35,9 +35,6 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long,unsigned long, unsigned long) ;
#include <asm/syscalls_64.h>
Quoted 15 lines
arch/x86/um/user-offsets.c static char syscalls[] = { · line 20
Quoted 13 lines
arch/x86/um/user-offsets.c +0
-2@@ -15,8 +15,6 @@ static char syscalls[] = { };#else #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ static char syscalls[] = {#include <asm/syscalls_64.h> };I am really unhappy about this change.
syscalltbl.shis written so that other architectures can use it, and so I would really prefer for it to stay arch-neutral and encourage other arches to use it, too.
We could use a qualifier (ie. /x32 or /x32ptregs) instead.
--
Brian Gerst
Re: [tip:x86/asm] x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
Quoted 139 lines · Brian Gerst
Quoted 122 lines · H. Peter Anvin
Quoted 121 lines · tip-bot for Andy Lutomirski
Commit-ID:
32324ce15ea8cb4c8acc28acb2fd36fabf73e9dbGitweb:
http://git.kernel.org/tip/32324ce15ea8cb4c8acc28acb2fd36fabf73e9dbAuthor: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:22 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:37 +0100x86/syscalls: Remove
__SYSCALL_COMMONand__SYSCALL_X32The common/64/x32 distinction has no effect other than determining which kernels actually support the syscall. Move the logic into
syscalltbl.sh.Signed-off-by Andy Lutomirski <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Borislav Petkov <[email protected]>Cc Brian Gerst <[email protected]>Cc Denys Vlasenko <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc H. Peter Anvin <[email protected]>Cc Linus Torvalds <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Thomas Gleixner <[email protected]>Link:
http://lkml.kernel.org/r/58d4a95f40e43b894f93288b4a3633963d0ee22e.1454022279.git.luto@kernel.orgSigned-off-by Ingo Molnar <[email protected]>---
arch/x86/entry/syscall_64.c | 8 -------- arch/x86/entry/syscalls/syscalltbl.sh | 17 ++++++++++++++++- arch/x86/kernel/asm-offsets_64.c | 6 ------ arch/x86/um/sys_call_table_64.c | 3 --- arch/x86/um/user-offsets.c | 2 -- 5 files changed, 16 insertions(+), 20 deletions(-)arch/x86/entry/syscall_64.c +0
-8@@ -6,14 +6,6 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) - -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long,unsigned long, unsigned long) ;
#include <asm/syscalls_64.h>
#undef__SYSCALL_64arch/x86/entry/syscalls/syscalltbl.sh +0
-0@@ -18,6 +18,21 @@ emit() {grep '^[0-9]' "$in" | sort -n | (
while read nr abi name entry compat; do
abi=echo "$abi" | tr '[a-z]' '[A-Z]'arch/x86/entry/syscalls/syscalltbl.sh +16
-1- emit "$abi" "$nr" "$entry" "$compat" + if [ "$abi" == "COMMON" -o "$abi" == "64" ]; then + # COMMON is the same as 64, except that we don't expect X32 + # programs to use it. Our expectation has nothing to do with + # any generated code, so treat them the same. + emit 64 "$nr" "$entry" "$compat" + elif [ "$abi" == "X32" ]; then + # X32 is equivalent to 64 on an X32-compatible kernel. + echo "#ifdef CONFIG_X86_X32_ABI" + emit 64 "$nr" "$entry" "$compat" + echo "#endif" + elif [ "$abi" == "I386" ]; then + emit "$abi" "$nr" "$entry" "$compat" + else + echo "Unknown abi $abi" >&2 + exit 1 + fi done) > "$out"
arch/x86/kernel/asm-offsets_64.c +0
-6@@ -5,12 +5,6 @@#include <asm/ia32.h> #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) [nr] = 1, -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif static char syscalls_64[] = {#include <asm/syscalls_64.h> };arch/x86/um/sys_call_table_64.c +0
-3@@ -35,9 +35,6 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long,unsigned long, unsigned long) ;
#include <asm/syscalls_64.h>Quoted 15 lines
arch/x86/um/user-offsets.c static char syscalls[] = { · line 20
Quoted 13 lines
arch/x86/um/user-offsets.c +0
-2@@ -15,8 +15,6 @@ static char syscalls[] = { };#else #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ static char syscalls[] = {#include <asm/syscalls_64.h> };I am really unhappy about this change.
syscalltbl.shis written so that other architectures can use it, and so I would really prefer for it to stay arch-neutral and encourage other arches to use it, too.
Quoted 1 line
We could use a qualifier (ie. /x32 or /x32ptregs) instead.
No combinatorial explosion, please. We could use __SYSCALL(nr, sym, abi, qual), though.
--Andy
Re: [tip:x86/asm] x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
Quoted 17 lines · Andy Lutomirski
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
arch/x86/entry/syscalls/syscalltbl.sh +8
-0+ if [ "$abi" == "COMMON" -o "$abi" == "64" ]; then + # COMMON is the same as 64, except that we don't expect X32 + # programs to use it. Our expectation has nothing to do with + # any generated code, so treat them the same. + emit 64 "$nr" "$entry" "$compat" + elif [ "$abi" == "X32" ]; then + # X32 is equivalent to 64 on an X32-compatible kernel. + echo "#ifdef CONFIG_X86_X32_ABI"Quoted 8 lines
Quoted 8 lines
arch/x86/entry/syscalls/syscalltbl.sh emit() { · line 36
Quoted 8 lines
arch/x86/entry/syscalls/syscalltbl.sh +8
-0+ emit 64 "$nr" "$entry" "$compat" + echo "#endif" + elif [ "$abi" == "I386" ]; then + emit "$abi" "$nr" "$entry" "$compat" + else + echo "Unknown abi $abi" >&2 + exit 1 + fi
Quoted 2 lines
No combinatorial explosion, please. We could use
__SYSCALL(nr, sym, abi, qual), though.
Mind fixing it, so that we get back the arch-neutral property?
Thanks,
Ingo
Re: [tip:x86/asm] x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
Quoted 20 lines · Ingo Molnar
Quoted 16 lines · Andy Lutomirski
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
arch/x86/entry/syscalls/syscalltbl.sh +16
-0+ if [ "$abi" == "COMMON" -o "$abi" == "64" ]; then + # COMMON is the same as 64, except that we don't expect X32 + # programs to use it. Our expectation has nothing to do with + # any generated code, so treat them the same. + emit 64 "$nr" "$entry" "$compat" + elif [ "$abi" == "X32" ]; then + # X32 is equivalent to 64 on an X32-compatible kernel. + echo "#ifdef CONFIG_X86_X32_ABI" + emit 64 "$nr" "$entry" "$compat" + echo "#endif" + elif [ "$abi" == "I386" ]; then + emit "$abi" "$nr" "$entry" "$compat" + else + echo "Unknown abi $abi" >&2 + exit 1 + fi
Quoted 5 lines
Quoted 2 lines
No combinatorial explosion, please. We could use
__SYSCALL(nr, sym, abi, qual), though.Mind fixing it, so that we get back the arch-neutral property?
I need some guidance as to the goal to do a good job.
In the version in -tip, I have this thing:
if [ "$abi" == "64" -a -n "$compat" ]; then
echo "a compat entry for a 64-bit syscall makes no sense" >&2 exit 1
fi
Moving that outside the script will either be impossible or an exercise in really awful C preprocessor hacks. We could keep that under the theory that it's arch-neutral.
It might be nice to add a similar warning that a compat entry for an x32 syscall makes so sense. That's a little less arch-neutral, although it wouldn't be actively harmful on any architecture, since "x32" wouldn't occur in the first place.
Other than that, I could add a little header called syscall_abi_mapping.h containing something like:
#ifndef __SYSCALL_ABI_MAPPING_H
#define __SYSCALL_ABI_MAPPING_H
#ifdef CONFIG_X86_32
/* Only I386 entries should ever be compiled into 32-bit kernels. */ #define __SYSCALL_ABI_I386(nr, entry, qual, compat, compat_qual) __SYSCALL_I386(nr, entry, qual)
#else
/* I386 entries on 64-bit kernels use the compat entry point. */ #define __SYSCALL_ABI_I386(nr, entry, qual, compat, compat_qual) __SYSCALL_I386(nr, compat, compat_qual)
#define __SYSCALL_ABI_common(nr, entry, compat, qual) #define __SYSCALL_ABI_64(nr, entry, qual, compat, compat_qual) __SYSCALL_64(nr, entry, qual)
#ifdef CONFIG_X86_X32
#define __SYSCALL_ABI_x32(nr, entry, qual, compat, compat_qual) __SYSCALL_64(nr, entry, qual)
#else
#define __SYSCALL_ABI_x32(nr, entry, qual, compat, compat_qual) __SYSCALL_64(nr, entry, qual)
#endif
#endif
#endif
and teach syscalltbl.sh to emit #include <asm/syscall_abi_mapping.h> at the beginning of syscalls_32.h and syscalls_64.h and to reference those macros.
hpa, would that meet your requirements?
IMO this is quite a bit messier than the code in -tip, and I'm honestly not convinced it's an improvement.
--Andy
H. Peter Anvin
reviewerRe: [tip:x86/asm] x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
Quoted 84 lines · Andy Lutomirski
Quoted 5 lines · Ingo Molnar
Quoted 2 lines · Andy Lutomirski
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ if [ "$abi" == "COMMON" -o "$abi" == "64" ]; then + # COMMON is the same as 64, except that we don'texpect X32
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ # programs to use it. Our expectation has nothing to
do with
Quoted 14 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
arch/x86/entry/syscalls/syscalltbl.sh +13
-0+ # any generated code, so treat them the same. + emit 64 "$nr" "$entry" "$compat" + elif [ "$abi" == "X32" ]; then + # X32 is equivalent to 64 on an X32-compatible kernel. + echo "#ifdef CONFIG_X86_X32_ABI" + emit 64 "$nr" "$entry" "$compat" + echo "#endif" + elif [ "$abi" == "I386" ]; then + emit "$abi" "$nr" "$entry" "$compat" + else + echo "Unknown abi $abi" >&2 + exit 1 + fiQuoted 5 lines
Quoted 2 lines
No combinatorial explosion, please. We could use
__SYSCALL(nr, sym, abi, qual), though.Mind fixing it, so that we get back the arch-neutral property?
I need some guidance as to the goal to do a good job.
In the version in -tip, I have this thing:
if [ "$abi" == "64" -a -n "$compat" ]; then
echo "a compat entry for a 64-bit syscall makes no sense" >&2 exit 1
fiMoving that outside the script will either be impossible or an exercise in really awful C preprocessor hacks. We could keep that under the theory that it's arch-neutral.
It might be nice to add a similar warning that a compat entry for an x32 syscall makes so sense. That's a little less arch-neutral, although it wouldn't be actively harmful on any architecture, since "x32" wouldn't occur in the first place.
Other than that, I could add a little header called
syscall_abi_mapping.hcontaining something like:#ifndef
__SYSCALL_ABI_MAPPING_H
#define__SYSCALL_ABI_MAPPING_H#ifdef
CONFIG_X86_32/* Only I386 entries should ever be compiled into 32-bit kernels. */ #define
__SYSCALL_ABI_I386(nr, entry, qual, compat, compat_qual)__SYSCALL_I386(nr, entry, qual)#else
/* I386 entries on 64-bit kernels use the compat entry point. */ #define
__SYSCALL_ABI_I386(nr, entry, qual, compat, compat_qual)__SYSCALL_I386(nr, compat, compat_qual)#define
__SYSCALL_ABI_common(nr, entry, compat, qual)#define__SYSCALL_ABI_64(nr, entry, qual, compat, compat_qual)__SYSCALL_64(nr, entry, qual)
#ifdefCONFIG_X86_X32
#define__SYSCALL_ABI_x32(nr, entry, qual, compat, compat_qual)__SYSCALL_64(nr, entry, qual)
#else
#define__SYSCALL_ABI_x32(nr, entry, qual, compat, compat_qual)__SYSCALL_64(nr, entry, qual)
#endif#endif
#endif
and teach
syscalltbl.shto emit #include <asm/syscall_abi_mapping.h> at the beginning ofsyscalls_32.handsyscalls_64.hand to reference those macros.
Quoted 6 lines
hpa, would that meet your requirements?
IMO this is quite a bit messier than the code in -tip, and I'm honestly not convinced it's an improvement.
--Andy
Something like that... however, I can't look at in detail right now.
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.
H. Peter Anvin
reviewerRe: [tip:x86/asm] x86/syscalls: Remove __SYSCALL_COMMON and __SYSCALL_X32
In reply to Brian GerstQuoted 143 lines · Brian Gerst
Quoted 1 line · H. Peter Anvin
On January 29, 2016 3:34:44 AM PST, tip-bot for Andy Lutomirski
Quoted 35 lines · <[email protected]> wrote:
Quoted 35 lines
Commit-ID:
32324ce15ea8cb4c8acc28acb2fd36fabf73e9dbGitweb:
http://git.kernel.org/tip/32324ce15ea8cb4c8acc28acb2fd36fabf73e9dbAuthor: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:22 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:37 +0100x86/syscalls: Remove
__SYSCALL_COMMONand__SYSCALL_X32The common/64/x32 distinction has no effect other than determining which kernels actually support the syscall. Move the logic into
syscalltbl.sh.Signed-off-by Andy Lutomirski <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Borislav Petkov <[email protected]>Cc Brian Gerst <[email protected]>Cc Denys Vlasenko <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc H. Peter Anvin <[email protected]>Cc Linus Torvalds <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Thomas Gleixner <[email protected]>Link:
http://lkml.kernel.org/r/58d4a95f40e43b894f93288b4a3633963d0ee22e.1454022279.git.luto@kernel.orgSigned-off-by Ingo Molnar <[email protected]>---
arch/x86/entry/syscall_64.c | 8 -------- arch/x86/entry/syscalls/syscalltbl.sh | 17 ++++++++++++++++- arch/x86/kernel/asm-offsets_64.c | 6 ------ arch/x86/um/sys_call_table_64.c | 3 --- arch/x86/um/user-offsets.c | 2 -- 5 files changed, 16 insertions(+), 20 deletions(-)Quoted 1 line
Review
Quoted 1 line
(quoted diff) +0
-0
b/arch/x86/entry/syscall_64.cQuoted 7 lines
Quoted 7 lines
index 41283d2..974fd89 100644
arch/x86/entry/syscall_64.c +0
-0@@ -6,14 +6,6 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> Quoted 5 lines
arch/x86/entry/syscall_64.c line 13
Quoted 5 lines
arch/x86/entry/syscall_64.c +0
-4-#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) - -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) __SYSCALL_64(nr, sym,compat)
Quoted 13 lines
Quoted 13 lines
arch/x86/entry/syscall_64.c +0
-4-#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long,unsigned long, unsigned long) ;
#include <asm/syscalls_64.h>
#undef__SYSCALL_64arch/x86/entry/syscalls/syscalltbl.sh +0
-1--- a/arch/x86/entry/syscalls/syscalltbl.shQuoted 8 lines
arch/x86/entry/syscalls/syscalltbl.sh emit() {
Quoted 8 lines
+++
b/arch/x86/entry/syscalls/syscalltbl.sharch/x86/entry/syscalls/syscalltbl.sh +0
-0@@ -18,6 +18,21 @@ emit() {grep '^[0-9]' "$in" | sort -n | (
while read nr abi name entry compat; do
abi=echo "$abi" | tr '[a-z]' '[A-Z]'arch/x86/entry/syscalls/syscalltbl.sh +2
-1- emit "$abi" "$nr" "$entry" "$compat" + if [ "$abi" == "COMMON" -o "$abi" == "64" ]; then + # COMMON is the same as 64, except that we don't expectX32
Quoted 1 line
Quoted 1 line
+ # programs to use it. Our expectation has nothing to do
with
Quoted 51 lines
Quoted 51 lines
arch/x86/entry/syscalls/syscalltbl.sh +13
-0+ # any generated code, so treat them the same. + emit 64 "$nr" "$entry" "$compat" + elif [ "$abi" == "X32" ]; then + # X32 is equivalent to 64 on an X32-compatible kernel. + echo "#ifdef CONFIG_X86_X32_ABI" + emit 64 "$nr" "$entry" "$compat" + echo "#endif" + elif [ "$abi" == "I386" ]; then + emit "$abi" "$nr" "$entry" "$compat" + else + echo "Unknown abi $abi" >&2 + exit 1 + fi done) > "$out"
arch/x86/kernel/asm-offsets_64.c +0
-6@@ -5,12 +5,6 @@#include <asm/ia32.h> #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#ifdef CONFIG_X86_X32_ABI -# define __SYSCALL_X32 (nr, sym, compat) [nr] = 1, -#else -# define __SYSCALL_X32 (nr, sym, compat)/* nothing */ -#endif static char syscalls_64[] = {#include <asm/syscalls_64.h> };arch/x86/um/sys_call_table_64.c +0
-3@@ -35,9 +35,6 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_COMMON (nr, sym, compat) __SYSCALL_64(nr, sym, compat) -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long,unsigned long, unsigned long) ;
#include <asm/syscalls_64.h>Quoted 15 lines
arch/x86/um/user-offsets.c static char syscalls[] = { · line 20
Quoted 13 lines
arch/x86/um/user-offsets.c +0
-2@@ -15,8 +15,6 @@ static char syscalls[] = { };#else #define __SYSCALL_64 (nr, sym, compat) [nr] = 1, -#define __SYSCALL_COMMON (nr, sym, compat) [nr] = 1, -#define __SYSCALL_X32 (nr, sym, compat)/* Not supported */ static char syscalls[] = {#include <asm/syscalls_64.h> };I am really unhappy about this change.
syscalltbl.shis written so
Quoted 7 lines
that other architectures can use it, and so I would really prefer for it to stay arch-neutral and encourage other arches to use it, too.
We could use a qualifier (ie. /x32 or /x32ptregs) instead.
--
Brian Gerst
I like that idea much better.
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.
Andy Lutomirski
maintainerPATCH v2 5/10 x86/syscalls: Move compat syscall entry handling into syscalltbl.sh
In reply to Andy LutomirskiRather than duplicating the compat entry handling in all consumers of syscalls_BITS.h, handle it directly in syscalltbl.sh. Now we generate entries in syscalls_32.h like:
__SYSCALL_I386(5, sys_open)__SYSCALL_I386(5, compat_sys_open)
and all of its consumers implicitly get the right entry point.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/entry/syscall_32.c | 10 ++-------- arch/x86/entry/syscall_64.c | 4 ++-- arch/x86/entry/syscalls/syscalltbl.sh | 22 ++++++++++++++++++---- arch/x86/kernel/asm-offsets_32.c | 2 +- arch/x86/kernel/asm-offsets_64.c | 4 ++-- arch/x86/um/sys_call_table_32.c | 4 ++-- arch/x86/um/sys_call_table_64.c | 4 ++-- arch/x86/um/user-offsets.c | 4 ++-- 8 files changed, 31 insertions(+), 23 deletions(-)
arch/x86/entry/syscall_32.c +2 -8
@@ -6,17 +6,11 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#ifdef CONFIG_IA32_EMULATION -#define SYM (sym, compat) compat -#else -#define SYM (sym, compat) sym -#endif - -#define __SYSCALL_I386 (nr, sym, compat) extern asmlinkage long SYM(sym, compat)(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_I386 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_32.h> #undef __SYSCALL_I386 -#define __SYSCALL_I386 (nr, sym, compat) [nr] = SYM(sym, compat), +#define __SYSCALL_I386 (nr, sym) [nr] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscall_64.c +2 -2
@@ -6,11 +6,11 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym, compat) [nr] = sym, +#define __SYSCALL_64 (nr, sym) [nr] = sym,extern long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscalls/syscalltbl.sh +18 -4
@@ -8,10 +8,24 @@ emit() { nr="$2" entry="$3" compat="$4" - if [ -n "$compat" ]; then - echo "__SYSCALL_${abi}($nr, $entry, $compat)" - elif [ -n "$entry" ]; then - echo "__SYSCALL_${abi}($nr, $entry, $entry)" + + if [ "$abi" == "64" -a -n "$compat" ]; then + echo "a compat entry for a 64-bit syscall makes no sense" >&2 + exit 1 + fi + + if [ -z "$compat" ]; then + if [ -n "$entry" ]; then + echo "__SYSCALL_${abi}($nr, $entry)" + fi + else + echo "#ifdef CONFIG_X86_32" + if [ -n "$entry" ]; then + echo "__SYSCALL_${abi}($nr, $entry)" + fi + echo "#else" + echo "__SYSCALL_${abi}($nr, $compat)" + echo "#endif" fi }
arch/x86/kernel/asm-offsets_32.c +1 -1
@@ -7,7 +7,7 @@#include <linux/lguest.h> #include "../../../drivers/lguest/lg.h" -#define __SYSCALL_I386 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_I386 (nr, sym) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_32.h> };
arch/x86/kernel/asm-offsets_64.c +2 -2
@@ -4,11 +4,11 @@#include <asm/ia32.h> -#define __SYSCALL_64 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_64 (nr, sym) [nr] = 1, static char syscalls_64[] = {#include <asm/syscalls_64.h> }; -#define __SYSCALL_I386 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_I386 (nr, sym) [nr] = 1, static char syscalls_ia32[] = {#include <asm/syscalls_32.h> };
arch/x86/um/sys_call_table_32.c +2 -2
@@ -25,11 +25,11 @@#define old_mmap sys_old_mmap -#define __SYSCALL_I386 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_I386 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_32.h> #undef __SYSCALL_I386 -#define __SYSCALL_I386 (nr, sym, compat) [ nr ] = sym, +#define __SYSCALL_I386 (nr, sym) [ nr ] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/um/sys_call_table_64.c +2 -2
@@ -35,11 +35,11 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym, compat) [ nr ] = sym, +#define __SYSCALL_64 (nr, sym) [ nr ] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/um/user-offsets.c +2 -2
@@ -9,12 +9,12 @@#include <asm/types.h> #ifdef __i386__ -#define __SYSCALL_I386 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_I386 (nr, sym) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_32.h> };#else -#define __SYSCALL_64 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_64 (nr, sym) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_64.h> };
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] x86/syscalls: Move compat syscall entry handling into syscalltbl.sh
x86/syscalls: Move compat syscall entry handling into syscalltbl.sh
Commit-ID: 3e65654e3db6df6aba9c5b895f8b8e6a8d8eb508 Gitweb: http://git.kernel.org/tip/3e65654e3db6df6aba9c5b895f8b8e6a8d8eb508 Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:23 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:37 +0100
x86/syscalls: Move compat syscall entry handling into syscalltbl.sh
Rather than duplicating the compat entry handling in all consumers of syscalls_BITS.h, handle it directly in syscalltbl.sh. Now we generate entries in syscalls_32.h like:
__SYSCALL_I386(5, sys_open)__SYSCALL_I386(5, compat_sys_open)
and all of its consumers implicitly get the right entry point.
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/b7c2b501dc0e6e43050e916b95807c3e2e16e9bb.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/entry/syscall_32.c | 10 ++-------- arch/x86/entry/syscall_64.c | 4 ++-- arch/x86/entry/syscalls/syscalltbl.sh | 22 ++++++++++++++++++---- arch/x86/kernel/asm-offsets_32.c | 2 +- arch/x86/kernel/asm-offsets_64.c | 4 ++-- arch/x86/um/sys_call_table_32.c | 4 ++-- arch/x86/um/sys_call_table_64.c | 4 ++-- arch/x86/um/user-offsets.c | 4 ++-- 8 files changed, 31 insertions(+), 23 deletions(-)
arch/x86/entry/syscall_32.c +2 -8
@@ -6,17 +6,11 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#ifdef CONFIG_IA32_EMULATION -#define SYM (sym, compat) compat -#else -#define SYM (sym, compat) sym -#endif - -#define __SYSCALL_I386 (nr, sym, compat) extern asmlinkage long SYM(sym, compat)(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_I386 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_32.h> #undef __SYSCALL_I386 -#define __SYSCALL_I386 (nr, sym, compat) [nr] = SYM(sym, compat), +#define __SYSCALL_I386 (nr, sym) [nr] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscall_64.c +2 -2
@@ -6,11 +6,11 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym, compat) [nr] = sym, +#define __SYSCALL_64 (nr, sym) [nr] = sym,extern long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscalls/syscalltbl.sh +18 -4
@@ -8,10 +8,24 @@ emit() { nr="$2" entry="$3" compat="$4" - if [ -n "$compat" ]; then - echo "__SYSCALL_${abi}($nr, $entry, $compat)" - elif [ -n "$entry" ]; then - echo "__SYSCALL_${abi}($nr, $entry, $entry)" + + if [ "$abi" == "64" -a -n "$compat" ]; then + echo "a compat entry for a 64-bit syscall makes no sense" >&2 + exit 1 + fi + + if [ -z "$compat" ]; then + if [ -n "$entry" ]; then + echo "__SYSCALL_${abi}($nr, $entry)" + fi + else + echo "#ifdef CONFIG_X86_32" + if [ -n "$entry" ]; then + echo "__SYSCALL_${abi}($nr, $entry)" + fi + echo "#else" + echo "__SYSCALL_${abi}($nr, $compat)" + echo "#endif" fi }
arch/x86/kernel/asm-offsets_32.c +1 -1
@@ -7,7 +7,7 @@#include <linux/lguest.h> #include "../../../drivers/lguest/lg.h" -#define __SYSCALL_I386 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_I386 (nr, sym) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_32.h> };
arch/x86/kernel/asm-offsets_64.c +2 -2
@@ -4,11 +4,11 @@#include <asm/ia32.h> -#define __SYSCALL_64 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_64 (nr, sym) [nr] = 1, static char syscalls_64[] = {#include <asm/syscalls_64.h> }; -#define __SYSCALL_I386 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_I386 (nr, sym) [nr] = 1, static char syscalls_ia32[] = {#include <asm/syscalls_32.h> };
arch/x86/um/sys_call_table_32.c +2 -2
@@ -25,11 +25,11 @@#define old_mmap sys_old_mmap -#define __SYSCALL_I386 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_I386 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_32.h> #undef __SYSCALL_I386 -#define __SYSCALL_I386 (nr, sym, compat) [ nr ] = sym, +#define __SYSCALL_I386 (nr, sym) [ nr ] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/um/sys_call_table_64.c +2 -2
@@ -35,11 +35,11 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_64 (nr, sym, compat) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym, compat) [ nr ] = sym, +#define __SYSCALL_64 (nr, sym) [ nr ] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/um/user-offsets.c +2 -2
@@ -9,12 +9,12 @@#include <asm/types.h> #ifdef __i386__ -#define __SYSCALL_I386 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_I386 (nr, sym) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_32.h> };#else -#define __SYSCALL_64 (nr, sym, compat) [nr] = 1, +#define __SYSCALL_64 (nr, sym) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_64.h> };
Andy Lutomirski
maintainerPATCH v2 6/10 x86/syscalls: Add syscall entry qualifiers
In reply to Andy LutomirskiThis will let us specify something like sys_xyz/foo instead of sys_xyz in the syscall table, where the foo conveys some information to the C code.
The intent is to allow things like sys_execve/ptregs to indicate that sys_execve touches pt_regs.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/entry/syscall_32.c | 4 ++-- arch/x86/entry/syscall_64.c | 4 ++-- arch/x86/entry/syscalls/syscalltbl.sh | 19 ++++++++++++++++--- arch/x86/kernel/asm-offsets_32.c | 2 +- arch/x86/kernel/asm-offsets_64.c | 4 ++-- arch/x86/um/sys_call_table_32.c | 4 ++-- arch/x86/um/sys_call_table_64.c | 4 ++-- arch/x86/um/user-offsets.c | 4 ++-- 8 files changed, 29 insertions(+), 16 deletions(-)
arch/x86/entry/syscall_32.c +2 -2
@@ -6,11 +6,11 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_I386 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_I386 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_32.h> #undef __SYSCALL_I386 -#define __SYSCALL_I386 (nr, sym) [nr] = sym, +#define __SYSCALL_I386 (nr, sym, qual) [nr] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscall_64.c +2 -2
@@ -6,11 +6,11 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_64 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym) [nr] = sym, +#define __SYSCALL_64 (nr, sym, qual) [nr] = sym,extern long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscalls/syscalltbl.sh +16 -3
@@ -3,6 +3,19 @@ in="$1" out="$2" +syscall_macro() { + abi="$1" + nr="$2" + entry="$3" + + # Entry can be either just a function name or "function/qualifier" + real_entry="${entry%%/*}" + qualifier="${entry:${#real_entry}}" # Strip the function name + qualifier="${qualifier:1}" # Strip the slash, if any + + echo "__SYSCALL_${abi}($nr, $real_entry, $qualifier)" +} + emit() { abi="$1" nr="$2"
@@ -16,15 +29,15 @@ emit() { if [ -z "$compat" ]; then if [ -n "$entry" ]; then - echo "__SYSCALL_${abi}($nr, $entry)" + syscall_macro "$abi" "$nr" "$entry" fi else echo "#ifdef CONFIG_X86_32" if [ -n "$entry" ]; then - echo "__SYSCALL_${abi}($nr, $entry)" + syscall_macro "$abi" "$nr" "$entry" fi echo "#else" - echo "__SYSCALL_${abi}($nr, $compat)" + syscall_macro "$abi" "$nr" "$compat" echo "#endif" fi }
arch/x86/kernel/asm-offsets_32.c +1 -1
@@ -7,7 +7,7 @@#include <linux/lguest.h> #include "../../../drivers/lguest/lg.h" -#define __SYSCALL_I386 (nr, sym) [nr] = 1, +#define __SYSCALL_I386 (nr, sym, qual) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_32.h> };
arch/x86/kernel/asm-offsets_64.c +2 -2
@@ -4,11 +4,11 @@#include <asm/ia32.h> -#define __SYSCALL_64 (nr, sym) [nr] = 1, +#define __SYSCALL_64 (nr, sym, qual) [nr] = 1, static char syscalls_64[] = {#include <asm/syscalls_64.h> }; -#define __SYSCALL_I386 (nr, sym) [nr] = 1, +#define __SYSCALL_I386 (nr, sym, qual) [nr] = 1, static char syscalls_ia32[] = {#include <asm/syscalls_32.h> };
arch/x86/um/sys_call_table_32.c +2 -2
@@ -25,11 +25,11 @@#define old_mmap sys_old_mmap -#define __SYSCALL_I386 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_I386 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_32.h> #undef __SYSCALL_I386 -#define __SYSCALL_I386 (nr, sym) [ nr ] = sym, +#define __SYSCALL_I386 (nr, sym, qual) [ nr ] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/um/sys_call_table_64.c +2 -2
@@ -35,11 +35,11 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_64 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym) [ nr ] = sym, +#define __SYSCALL_64 (nr, sym, qual) [ nr ] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/um/user-offsets.c +2 -2
@@ -9,12 +9,12 @@#include <asm/types.h> #ifdef __i386__ -#define __SYSCALL_I386 (nr, sym) [nr] = 1, +#define __SYSCALL_I386 (nr, sym, qual) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_32.h> };#else -#define __SYSCALL_64 (nr, sym) [nr] = 1, +#define __SYSCALL_64 (nr, sym, qual) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_64.h> };
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] x86/syscalls: Add syscall entry qualifiers
x86/syscalls: Add syscall entry qualifiers
Commit-ID: cfcbadb49dabb05efa23e1a0f95f3391c0a815bc Gitweb: http://git.kernel.org/tip/cfcbadb49dabb05efa23e1a0f95f3391c0a815bc Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:24 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:38 +0100
x86/syscalls: Add syscall entry qualifiers
This will let us specify something like 'sys_xyz/foo' instead of 'sys_xyz' in the syscall table, where the 'foo' qualifier conveys some extra information to the C code.
The intent is to allow things like sys_execve/ptregs to indicate that sys_execve() touches pt_regs.
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/2de06e33dce62556b3ec662006fcb295504e296e.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/entry/syscall_32.c | 4 ++-- arch/x86/entry/syscall_64.c | 4 ++-- arch/x86/entry/syscalls/syscalltbl.sh | 19 ++++++++++++++++--- arch/x86/kernel/asm-offsets_32.c | 2 +- arch/x86/kernel/asm-offsets_64.c | 4 ++-- arch/x86/um/sys_call_table_32.c | 4 ++-- arch/x86/um/sys_call_table_64.c | 4 ++-- arch/x86/um/user-offsets.c | 4 ++-- 8 files changed, 29 insertions(+), 16 deletions(-)
arch/x86/entry/syscall_32.c +2 -2
@@ -6,11 +6,11 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_I386 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_I386 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_32.h> #undef __SYSCALL_I386 -#define __SYSCALL_I386 (nr, sym) [nr] = sym, +#define __SYSCALL_I386 (nr, sym, qual) [nr] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscall_64.c +2 -2
@@ -6,11 +6,11 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_64 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym) [nr] = sym, +#define __SYSCALL_64 (nr, sym, qual) [nr] = sym,extern long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscalls/syscalltbl.sh +16 -3
@@ -3,6 +3,19 @@ in="$1" out="$2" +syscall_macro() { + abi="$1" + nr="$2" + entry="$3" + + # Entry can be either just a function name or "function/qualifier" + real_entry="${entry%%/*}" + qualifier="${entry:${#real_entry}}" # Strip the function name + qualifier="${qualifier:1}" # Strip the slash, if any + + echo "__SYSCALL_${abi}($nr, $real_entry, $qualifier)" +} + emit() { abi="$1" nr="$2"
@@ -16,15 +29,15 @@ emit() { if [ -z "$compat" ]; then if [ -n "$entry" ]; then - echo "__SYSCALL_${abi}($nr, $entry)" + syscall_macro "$abi" "$nr" "$entry" fi else echo "#ifdef CONFIG_X86_32" if [ -n "$entry" ]; then - echo "__SYSCALL_${abi}($nr, $entry)" + syscall_macro "$abi" "$nr" "$entry" fi echo "#else" - echo "__SYSCALL_${abi}($nr, $compat)" + syscall_macro "$abi" "$nr" "$compat" echo "#endif" fi }
arch/x86/kernel/asm-offsets_32.c +1 -1
@@ -7,7 +7,7 @@#include <linux/lguest.h> #include "../../../drivers/lguest/lg.h" -#define __SYSCALL_I386 (nr, sym) [nr] = 1, +#define __SYSCALL_I386 (nr, sym, qual) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_32.h> };
arch/x86/kernel/asm-offsets_64.c +2 -2
@@ -4,11 +4,11 @@#include <asm/ia32.h> -#define __SYSCALL_64 (nr, sym) [nr] = 1, +#define __SYSCALL_64 (nr, sym, qual) [nr] = 1, static char syscalls_64[] = {#include <asm/syscalls_64.h> }; -#define __SYSCALL_I386 (nr, sym) [nr] = 1, +#define __SYSCALL_I386 (nr, sym, qual) [nr] = 1, static char syscalls_ia32[] = {#include <asm/syscalls_32.h> };
arch/x86/um/sys_call_table_32.c +2 -2
@@ -25,11 +25,11 @@#define old_mmap sys_old_mmap -#define __SYSCALL_I386 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_I386 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_32.h> #undef __SYSCALL_I386 -#define __SYSCALL_I386 (nr, sym) [ nr ] = sym, +#define __SYSCALL_I386 (nr, sym, qual) [ nr ] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/um/sys_call_table_64.c +2 -2
@@ -35,11 +35,11 @@#define stub_execveat sys_execveat#define stub_rt_sigreturn sys_rt_sigreturn -#define __SYSCALL_64 (nr, sym) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ;#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym) [ nr ] = sym, +#define __SYSCALL_64 (nr, sym, qual) [ nr ] = sym,extern asmlinkage long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/um/user-offsets.c +2 -2
@@ -9,12 +9,12 @@#include <asm/types.h> #ifdef __i386__ -#define __SYSCALL_I386 (nr, sym) [nr] = 1, +#define __SYSCALL_I386 (nr, sym, qual) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_32.h> };#else -#define __SYSCALL_64 (nr, sym) [nr] = 1, +#define __SYSCALL_64 (nr, sym, qual) [nr] = 1, static char syscalls[] = {#include <asm/syscalls_64.h> };
Andy Lutomirski
maintainerPATCH v2 7/10 x86/entry/64: Always run ptregs-using syscalls on the slow path
In reply to Andy Lutomirski64-bit syscalls currently have an optimization in which they are called with partial pt_regs. A small handful require full pt_regs.
In the 32-bit and compat cases, I cleaned this up by forcing full pt_regs for all syscalls. The performance hit doesn't really matter.
I want to clean up the 64-bit case as well, but I don't want to hurt fast path performance. To do that, I want to force the syscalls that use pt_regs onto the slow path. This will enable us to make slow path syscalls be real ABI-compliant C functions.
Use the new syscall entry qualification machinery for this. stub_clone is now stub_clone/ptregs.
The next patch will eliminate the stubs, and we'll just have sys_clone/ptregs.
As of this patch, two-phase entry tracing is no longer used. It has served its purpose (namely a huge speedup on some workloads prior to more general opportunistic SYSRET support), and once the dust settles I'll send patches to back it out.
The implementation is heavily based on a patch from Brian Gerst [1].
[1] http://lkml.kernel.org/g/[email protected]
Signed-off-by Brian Gerst <[email protected]>
Cc the arch/x86 maintainers <[email protected]>
Cc Linux Kernel Mailing List <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Frédéric Weisbecker <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Linus Torvalds <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/entry/entry_64.S | 56 +++++++++++++++++++++++++--------- arch/x86/entry/syscall_64.c | 7 +++-- arch/x86/entry/syscalls/syscall_64.tbl | 16 +++++----- 3 files changed, 55 insertions(+), 24 deletions(-)
arch/x86/entry/entry_64.S +42 -14
@@ -182,7 +182,15 @@ entry_SYSCALL_64_fastpath: #endif ja 1f /* return -ENOSYS (already in pt_regs->ax) */ movq %r10, %rcx + + /* + * This call instruction is handled specially in stub_ptregs_64. + * It might end up jumping to the slow path. If it jumps, RAX is + * clobbered. + */ call *sys_call_table(, %rax, 8) +.Lentry_SYSCALL_64_after_fastpath_call: + movq %rax, RAX(%rsp) 1: /*
@@ -235,25 +243,13 @@ GLOBAL(int_ret_from_sys_call_irqs_off) /* Do syscall entry tracing */ tracesys: - movq %rsp, %rdi - movl $AUDIT_ARCH_X86_64, %esi - call syscall_trace_enter_phase1 - test %rax, %rax - jnz tracesys_phase2 /* if needed, run the slow path */ - RESTORE_C_REGS_EXCEPT_RAX /* else restore clobbered regs */ - movq ORIG_RAX(%rsp), %rax - jmp entry_SYSCALL_64_fastpath /* and return to the fast path */ - -tracesys_phase2: SAVE_EXTRA_REGS movq %rsp, %rdi - movl $AUDIT_ARCH_X86_64, %esi - movq %rax, %rdx - call syscall_trace_enter_phase2 + call syscall_trace_enter /* * Reload registers from stack in case ptrace changed them. - * We don't reload %rax because syscall_trace_entry_phase2() returned + * We don't reload %rax because syscall_trace_enter() returned * the value it wants us to use in the table lookup. */ RESTORE_C_REGS_EXCEPT_RAX
@@ -355,6 +351,38 @@ opportunistic_sysret_failed: jmp restore_c_regs_and_iret END(entry_SYSCALL_64) +ENTRY(stub_ptregs_64) + /* + * Syscalls marked as needing ptregs land here. + * If we are on the fast path, we need to save the extra regs. + * If we are on the slow path, the extra regs are already saved. + * + * RAX stores a pointer to the C function implementing the syscall. + */ + cmpq $.Lentry_SYSCALL_64_after_fastpath_call, (%rsp) + jne 1f + + /* Called from fast path -- pop return address and jump to slow path */ + popq %rax + jmp tracesys /* called from fast path */ + +1: + /* Called from C */ + jmp *%rax /* called from C */ +END(stub_ptregs_64) + +.macro ptregs_stub func +ENTRY(ptregs_\func) + leaq \func(%rip), %rax + jmp stub_ptregs_64 +END(ptregs_\func) +.endm + +/* Instantiate ptregs_stub for each ptregs-using syscall */ +#define __SYSCALL_64_QUAL_(sym) +#define __SYSCALL_64_QUAL_ptregs(sym) ptregs_stub sym +#define __SYSCALL_64(nr, sym, qual) __SYSCALL_64_QUAL_##qual(sym) +#include <asm/syscalls_64.h> .macro FORK_LIKE func ENTRY(stub_\func)
arch/x86/entry/syscall_64.c +5 -2
@@ -6,11 +6,14 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_64 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64_QUAL_ (sym) sym +#define __SYSCALL_64_QUAL_ptregs (sym) ptregs_##sym + +#define __SYSCALL_64 (nr, sym, qual) extern asmlinkage long __SYSCALL_64_QUAL_##qual(sym)(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long);#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym, qual) [nr] = sym, +#define __SYSCALL_64 (nr, sym, qual) [nr] = __SYSCALL_64_QUAL_##qual(sym),extern long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscalls/syscall_64.tbl +8 -8
@@ -21,7 +21,7 @@ 12 common brk sys_brk 13 64 rt_sigaction sys_rt_sigaction 14 common rt_sigprocmask sys_rt_sigprocmask -15 64 rt_sigreturn stub_rt_sigreturn +15 64 rt_sigreturn stub_rt_sigreturn/ptregs 16 64 ioctl sys_ioctl 17 common pread64 sys_pread64 18 common pwrite64 sys_pwrite64
@@ -62,10 +62,10 @@ 53 common socketpair sys_socketpair 54 64 setsockopt sys_setsockopt 55 64 getsockopt sys_getsockopt -56 common clone stub_clone -57 common fork stub_fork -58 common vfork stub_vfork -59 64 execve stub_execve +56 common clone stub_clone/ptregs +57 common fork stub_fork/ptregs +58 common vfork stub_vfork/ptregs +59 64 execve stub_execve/ptregs 60 common exit sys_exit 61 common wait4 sys_wait4 62 common kill sys_kill
@@ -328,7 +328,7 @@ 319 common memfd_create sys_memfd_create 320 common kexec_file_load sys_kexec_file_load 321 common bpf sys_bpf -322 64 execveat stub_execveat +322 64 execveat stub_execveat/ptregs 323 common userfaultfd sys_userfaultfd 324 common membarrier sys_membarrier 325 common mlock2 sys_mlock2
@@ -346,7 +346,7 @@ 517 x32 recvfrom compat_sys_recvfrom 518 x32 sendmsg compat_sys_sendmsg 519 x32 recvmsg compat_sys_recvmsg -520 x32 execve stub_x32_execve +520 x32 execve stub_x32_execve/ptregs 521 x32 ptrace compat_sys_ptrace 522 x32 rt_sigpending compat_sys_rt_sigpending 523 x32 rt_sigtimedwait compat_sys_rt_sigtimedwait
@@ -371,4 +371,4 @@ 542 x32 getsockopt compat_sys_getsockopt 543 x32 io_setup compat_sys_io_setup 544 x32 io_submit compat_sys_io_submit -545 x32 execveat stub_x32_execveat +545 x32 execveat stub_x32_execveat/ptregs
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] x86/entry/64: Always run ptregs-using syscalls on the slow path
x86/entry/64: Always run ptregs-using syscalls on the slow path
Commit-ID: 302f5b260c322696cbeb962a263a4d2d99864aed Gitweb: http://git.kernel.org/tip/302f5b260c322696cbeb962a263a4d2d99864aed Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:25 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:38 +0100
x86/entry/64: Always run ptregs-using syscalls on the slow path
64-bit syscalls currently have an optimization in which they are called with partial pt_regs. A small handful require full pt_regs.
In the 32-bit and compat cases, I cleaned this up by forcing full pt_regs for all syscalls. The performance hit doesn't really matter as the affected system calls are fundamentally heavy and this is the 32-bit compat case.
I want to clean up the 64-bit case as well, but I don't want to hurt fast path performance. To do that, I want to force the syscalls that use pt_regs onto the slow path. This will enable us to make slow path syscalls be real ABI-compliant C functions.
Use the new syscall entry qualification machinery for this. 'stub_clone' is now 'stub_clone/ptregs'.
The next patch will eliminate the stubs, and we'll just have 'sys_clone/ptregs'.
As of this patch, two-phase entry tracing is no longer used. It has served its purpose (namely a huge speedup on some workloads prior to more general opportunistic SYSRET support), and once the dust settles I'll send patches to back it out.
The implementation is heavily based on a patch from Brian Gerst:
http://lkml.kernel.org/g/[email protected]
Originally-From: Brian Gerst <[email protected]>
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Frédéric Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Linux Kernel Mailing List <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/b9beda88460bcefec6e7d792bd44eca9b760b0c4.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/entry/entry_64.S | 56 +++++++++++++++++++++++++--------- arch/x86/entry/syscall_64.c | 7 +++-- arch/x86/entry/syscalls/syscall_64.tbl | 16 +++++----- 3 files changed, 55 insertions(+), 24 deletions(-)
arch/x86/entry/entry_64.S +42 -14
@@ -182,7 +182,15 @@ entry_SYSCALL_64_fastpath: #endif ja 1f /* return -ENOSYS (already in pt_regs->ax) */ movq %r10, %rcx + + /* + * This call instruction is handled specially in stub_ptregs_64. + * It might end up jumping to the slow path. If it jumps, RAX is + * clobbered. + */ call *sys_call_table(, %rax, 8) +.Lentry_SYSCALL_64_after_fastpath_call: + movq %rax, RAX(%rsp) 1: /*
@@ -235,25 +243,13 @@ GLOBAL(int_ret_from_sys_call_irqs_off) /* Do syscall entry tracing */ tracesys: - movq %rsp, %rdi - movl $AUDIT_ARCH_X86_64, %esi - call syscall_trace_enter_phase1 - test %rax, %rax - jnz tracesys_phase2 /* if needed, run the slow path */ - RESTORE_C_REGS_EXCEPT_RAX /* else restore clobbered regs */ - movq ORIG_RAX(%rsp), %rax - jmp entry_SYSCALL_64_fastpath /* and return to the fast path */ - -tracesys_phase2: SAVE_EXTRA_REGS movq %rsp, %rdi - movl $AUDIT_ARCH_X86_64, %esi - movq %rax, %rdx - call syscall_trace_enter_phase2 + call syscall_trace_enter /* * Reload registers from stack in case ptrace changed them. - * We don't reload %rax because syscall_trace_entry_phase2() returned + * We don't reload %rax because syscall_trace_enter() returned * the value it wants us to use in the table lookup. */ RESTORE_C_REGS_EXCEPT_RAX
@@ -355,6 +351,38 @@ opportunistic_sysret_failed: jmp restore_c_regs_and_iret END(entry_SYSCALL_64) +ENTRY(stub_ptregs_64) + /* + * Syscalls marked as needing ptregs land here. + * If we are on the fast path, we need to save the extra regs. + * If we are on the slow path, the extra regs are already saved. + * + * RAX stores a pointer to the C function implementing the syscall. + */ + cmpq $.Lentry_SYSCALL_64_after_fastpath_call, (%rsp) + jne 1f + + /* Called from fast path -- pop return address and jump to slow path */ + popq %rax + jmp tracesys /* called from fast path */ + +1: + /* Called from C */ + jmp *%rax /* called from C */ +END(stub_ptregs_64) + +.macro ptregs_stub func +ENTRY(ptregs_\func) + leaq \func(%rip), %rax + jmp stub_ptregs_64 +END(ptregs_\func) +.endm + +/* Instantiate ptregs_stub for each ptregs-using syscall */ +#define __SYSCALL_64_QUAL_(sym) +#define __SYSCALL_64_QUAL_ptregs(sym) ptregs_stub sym +#define __SYSCALL_64(nr, sym, qual) __SYSCALL_64_QUAL_##qual(sym) +#include <asm/syscalls_64.h> .macro FORK_LIKE func ENTRY(stub_\func)
arch/x86/entry/syscall_64.c +5 -2
@@ -6,11 +6,14 @@#include <asm/asm-offsets.h> #include <asm/syscall.h> -#define __SYSCALL_64 (nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) ; +#define __SYSCALL_64_QUAL_ (sym) sym +#define __SYSCALL_64_QUAL_ptregs (sym) ptregs_##sym + +#define __SYSCALL_64 (nr, sym, qual) extern asmlinkage long __SYSCALL_64_QUAL_##qual(sym)(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long);#include <asm/syscalls_64.h> #undef __SYSCALL_64 -#define __SYSCALL_64 (nr, sym, qual) [nr] = sym, +#define __SYSCALL_64 (nr, sym, qual) [nr] = __SYSCALL_64_QUAL_##qual(sym),extern long sys_ni_syscall (unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long ,unsigned long );
arch/x86/entry/syscalls/syscall_64.tbl +8 -8
@@ -21,7 +21,7 @@ 12 common brk sys_brk 13 64 rt_sigaction sys_rt_sigaction 14 common rt_sigprocmask sys_rt_sigprocmask -15 64 rt_sigreturn stub_rt_sigreturn +15 64 rt_sigreturn stub_rt_sigreturn/ptregs 16 64 ioctl sys_ioctl 17 common pread64 sys_pread64 18 common pwrite64 sys_pwrite64
@@ -62,10 +62,10 @@ 53 common socketpair sys_socketpair 54 64 setsockopt sys_setsockopt 55 64 getsockopt sys_getsockopt -56 common clone stub_clone -57 common fork stub_fork -58 common vfork stub_vfork -59 64 execve stub_execve +56 common clone stub_clone/ptregs +57 common fork stub_fork/ptregs +58 common vfork stub_vfork/ptregs +59 64 execve stub_execve/ptregs 60 common exit sys_exit 61 common wait4 sys_wait4 62 common kill sys_kill
@@ -328,7 +328,7 @@ 319 common memfd_create sys_memfd_create 320 common kexec_file_load sys_kexec_file_load 321 common bpf sys_bpf -322 64 execveat stub_execveat +322 64 execveat stub_execveat/ptregs 323 common userfaultfd sys_userfaultfd 324 common membarrier sys_membarrier 325 common mlock2 sys_mlock2
@@ -346,7 +346,7 @@ 517 x32 recvfrom compat_sys_recvfrom 518 x32 sendmsg compat_sys_sendmsg 519 x32 recvmsg compat_sys_recvmsg -520 x32 execve stub_x32_execve +520 x32 execve stub_x32_execve/ptregs 521 x32 ptrace compat_sys_ptrace 522 x32 rt_sigpending compat_sys_rt_sigpending 523 x32 rt_sigtimedwait compat_sys_rt_sigtimedwait
@@ -371,4 +371,4 @@ 542 x32 getsockopt compat_sys_getsockopt 543 x32 io_setup compat_sys_io_setup 544 x32 io_submit compat_sys_io_submit -545 x32 execveat stub_x32_execveat +545 x32 execveat stub_x32_execveat/ptregs
Andy Lutomirski
maintainerPATCH v2 8/10 x86/entry/64: Call all native slow-path syscalls with full pt-regs
In reply to Andy LutomirskiThis removes all of the remaining asm syscall stubs except for stub_ptregs_64. Entries in the main syscall table are now all callable from C.
The resulting asm is every bit as ridiculous as it looks. The next few patches will clean it up. This patch is here to let reviewers rest their brains and for bisection.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/entry/entry_64.S | 79 +--------------------------------- arch/x86/entry/syscalls/syscall_64.tbl | 18 ++++---- 2 files changed, 10 insertions(+), 87 deletions(-)
arch/x86/entry/entry_64.S +1 -78
@@ -253,7 +253,6 @@ tracesys: * the value it wants us to use in the table lookup. */ RESTORE_C_REGS_EXCEPT_RAX - RESTORE_EXTRA_REGS #if __SYSCALL_MASK == ~0 cmpq $__NR_syscall_max, %rax #else
@@ -264,6 +263,7 @@ tracesys: movq %r10, %rcx /* fixup for C */ call *sys_call_table(, %rax, 8) movq %rax, RAX(%rsp) + RESTORE_EXTRA_REGS 1: /* Use IRET because user could have changed pt_regs->foo */
@@ -384,83 +384,6 @@ END(ptregs_\func) #define __SYSCALL_64(nr, sym, qual) __SYSCALL_64_QUAL_##qual(sym) #include <asm/syscalls_64.h> - .macro FORK_LIKE func -ENTRY(stub_\func) - SAVE_EXTRA_REGS 8 - jmp sys_\func -END(stub_\func) - .endm - - FORK_LIKE clone - FORK_LIKE fork - FORK_LIKE vfork - -ENTRY(stub_execve) - call sys_execve -return_from_execve: - testl %eax, %eax - jz 1f - /* exec failed, can use fast SYSRET code path in this case */ - ret -1: - /* must use IRET code path (pt_regs->cs may have changed) */ - addq $8, %rsp - ZERO_EXTRA_REGS - movq %rax, RAX(%rsp) - jmp int_ret_from_sys_call -END(stub_execve) -/* - * Remaining execve stubs are only 7 bytes long. - * ENTRY() often aligns to 16 bytes, which in this case has no benefits. - */ - .align 8 -GLOBAL(stub_execveat) - call sys_execveat - jmp return_from_execve -END(stub_execveat) - -#if defined(CONFIG_X86_X32_ABI) - .align 8 -GLOBAL(stub_x32_execve) - call compat_sys_execve - jmp return_from_execve -END(stub_x32_execve) - .align 8 -GLOBAL(stub_x32_execveat) - call compat_sys_execveat - jmp return_from_execve -END(stub_x32_execveat) -#endif - -/* - * sigreturn is special because it needs to restore all registers on return. - * This cannot be done with SYSRET, so use the IRET return path instead. - */ -ENTRY(stub_rt_sigreturn) - /* - * SAVE_EXTRA_REGS result is not normally needed: - * sigreturn overwrites all pt_regs->GPREGS. - * But sigreturn can fail (!), and there is no easy way to detect that. - * To make sure RESTORE_EXTRA_REGS doesn't restore garbage on error, - * we SAVE_EXTRA_REGS here. - */ - SAVE_EXTRA_REGS 8 - call sys_rt_sigreturn -return_from_stub: - addq $8, %rsp - RESTORE_EXTRA_REGS - movq %rax, RAX(%rsp) - jmp int_ret_from_sys_call -END(stub_rt_sigreturn) - -#ifdef CONFIG_X86_X32_ABI -ENTRY(stub_x32_rt_sigreturn) - SAVE_EXTRA_REGS 8 - call sys32_x32_rt_sigreturn - jmp return_from_stub -END(stub_x32_rt_sigreturn) -#endif - /* * A newly forked process directly context switches into this address. *
arch/x86/entry/syscalls/syscall_64.tbl +9 -9
@@ -21,7 +21,7 @@ 12 common brk sys_brk 13 64 rt_sigaction sys_rt_sigaction 14 common rt_sigprocmask sys_rt_sigprocmask -15 64 rt_sigreturn stub_rt_sigreturn/ptregs +15 64 rt_sigreturn sys_rt_sigreturn/ptregs 16 64 ioctl sys_ioctl 17 common pread64 sys_pread64 18 common pwrite64 sys_pwrite64
@@ -62,10 +62,10 @@ 53 common socketpair sys_socketpair 54 64 setsockopt sys_setsockopt 55 64 getsockopt sys_getsockopt -56 common clone stub_clone/ptregs -57 common fork stub_fork/ptregs -58 common vfork stub_vfork/ptregs -59 64 execve stub_execve/ptregs +56 common clone sys_clone/ptregs +57 common fork sys_fork/ptregs +58 common vfork sys_vfork/ptregs +59 64 execve sys_execve/ptregs 60 common exit sys_exit 61 common wait4 sys_wait4 62 common kill sys_kill
@@ -328,7 +328,7 @@ 319 common memfd_create sys_memfd_create 320 common kexec_file_load sys_kexec_file_load 321 common bpf sys_bpf -322 64 execveat stub_execveat/ptregs +322 64 execveat sys_execveat/ptregs 323 common userfaultfd sys_userfaultfd 324 common membarrier sys_membarrier 325 common mlock2 sys_mlock2
@@ -339,14 +339,14 @@ # for native 64-bit operation. # 512 x32 rt_sigaction compat_sys_rt_sigaction -513 x32 rt_sigreturn stub_x32_rt_sigreturn +513 x32 rt_sigreturn sys32_x32_rt_sigreturn 514 x32 ioctl compat_sys_ioctl 515 x32 readv compat_sys_readv 516 x32 writev compat_sys_writev 517 x32 recvfrom compat_sys_recvfrom 518 x32 sendmsg compat_sys_sendmsg 519 x32 recvmsg compat_sys_recvmsg -520 x32 execve stub_x32_execve/ptregs +520 x32 execve compat_sys_execve/ptregs 521 x32 ptrace compat_sys_ptrace 522 x32 rt_sigpending compat_sys_rt_sigpending 523 x32 rt_sigtimedwait compat_sys_rt_sigtimedwait
@@ -371,4 +371,4 @@ 542 x32 getsockopt compat_sys_getsockopt 543 x32 io_setup compat_sys_io_setup 544 x32 io_submit compat_sys_io_submit -545 x32 execveat stub_x32_execveat/ptregs +545 x32 execveat compat_sys_execveat/ptregs
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] x86/entry/64: Call all native slow-path syscalls with full pt-regs
x86/entry/64: Call all native slow-path syscalls with full pt-regs
Commit-ID: 46eabf06c04a6847a694a0c1413d4ac57e5b058a Gitweb: http://git.kernel.org/tip/46eabf06c04a6847a694a0c1413d4ac57e5b058a Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:26 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:38 +0100
x86/entry/64: Call all native slow-path syscalls with full pt-regs
This removes all of the remaining asm syscall stubs except for stub_ptregs_64. Entries in the main syscall table are now all callable from C.
The resulting asm is every bit as ridiculous as it looks. The next few patches will clean it up. This patch is here to let reviewers rest their brains and for bisection.
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/a6b3801be0d505d50aefabda02d3b93efbfc9c73.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/entry/entry_64.S | 79 +--------------------------------- arch/x86/entry/syscalls/syscall_64.tbl | 18 ++++---- 2 files changed, 10 insertions(+), 87 deletions(-)
arch/x86/entry/entry_64.S +1 -78
@@ -253,7 +253,6 @@ tracesys: * the value it wants us to use in the table lookup. */ RESTORE_C_REGS_EXCEPT_RAX - RESTORE_EXTRA_REGS #if __SYSCALL_MASK == ~0 cmpq $__NR_syscall_max, %rax #else
@@ -264,6 +263,7 @@ tracesys: movq %r10, %rcx /* fixup for C */ call *sys_call_table(, %rax, 8) movq %rax, RAX(%rsp) + RESTORE_EXTRA_REGS 1: /* Use IRET because user could have changed pt_regs->foo */
@@ -384,83 +384,6 @@ END(ptregs_\func) #define __SYSCALL_64(nr, sym, qual) __SYSCALL_64_QUAL_##qual(sym) #include <asm/syscalls_64.h> - .macro FORK_LIKE func -ENTRY(stub_\func) - SAVE_EXTRA_REGS 8 - jmp sys_\func -END(stub_\func) - .endm - - FORK_LIKE clone - FORK_LIKE fork - FORK_LIKE vfork - -ENTRY(stub_execve) - call sys_execve -return_from_execve: - testl %eax, %eax - jz 1f - /* exec failed, can use fast SYSRET code path in this case */ - ret -1: - /* must use IRET code path (pt_regs->cs may have changed) */ - addq $8, %rsp - ZERO_EXTRA_REGS - movq %rax, RAX(%rsp) - jmp int_ret_from_sys_call -END(stub_execve) -/* - * Remaining execve stubs are only 7 bytes long. - * ENTRY() often aligns to 16 bytes, which in this case has no benefits. - */ - .align 8 -GLOBAL(stub_execveat) - call sys_execveat - jmp return_from_execve -END(stub_execveat) - -#if defined(CONFIG_X86_X32_ABI) - .align 8 -GLOBAL(stub_x32_execve) - call compat_sys_execve - jmp return_from_execve -END(stub_x32_execve) - .align 8 -GLOBAL(stub_x32_execveat) - call compat_sys_execveat - jmp return_from_execve -END(stub_x32_execveat) -#endif - -/* - * sigreturn is special because it needs to restore all registers on return. - * This cannot be done with SYSRET, so use the IRET return path instead. - */ -ENTRY(stub_rt_sigreturn) - /* - * SAVE_EXTRA_REGS result is not normally needed: - * sigreturn overwrites all pt_regs->GPREGS. - * But sigreturn can fail (!), and there is no easy way to detect that. - * To make sure RESTORE_EXTRA_REGS doesn't restore garbage on error, - * we SAVE_EXTRA_REGS here. - */ - SAVE_EXTRA_REGS 8 - call sys_rt_sigreturn -return_from_stub: - addq $8, %rsp - RESTORE_EXTRA_REGS - movq %rax, RAX(%rsp) - jmp int_ret_from_sys_call -END(stub_rt_sigreturn) - -#ifdef CONFIG_X86_X32_ABI -ENTRY(stub_x32_rt_sigreturn) - SAVE_EXTRA_REGS 8 - call sys32_x32_rt_sigreturn - jmp return_from_stub -END(stub_x32_rt_sigreturn) -#endif - /* * A newly forked process directly context switches into this address. *
arch/x86/entry/syscalls/syscall_64.tbl +9 -9
@@ -21,7 +21,7 @@ 12 common brk sys_brk 13 64 rt_sigaction sys_rt_sigaction 14 common rt_sigprocmask sys_rt_sigprocmask -15 64 rt_sigreturn stub_rt_sigreturn/ptregs +15 64 rt_sigreturn sys_rt_sigreturn/ptregs 16 64 ioctl sys_ioctl 17 common pread64 sys_pread64 18 common pwrite64 sys_pwrite64
@@ -62,10 +62,10 @@ 53 common socketpair sys_socketpair 54 64 setsockopt sys_setsockopt 55 64 getsockopt sys_getsockopt -56 common clone stub_clone/ptregs -57 common fork stub_fork/ptregs -58 common vfork stub_vfork/ptregs -59 64 execve stub_execve/ptregs +56 common clone sys_clone/ptregs +57 common fork sys_fork/ptregs +58 common vfork sys_vfork/ptregs +59 64 execve sys_execve/ptregs 60 common exit sys_exit 61 common wait4 sys_wait4 62 common kill sys_kill
@@ -328,7 +328,7 @@ 319 common memfd_create sys_memfd_create 320 common kexec_file_load sys_kexec_file_load 321 common bpf sys_bpf -322 64 execveat stub_execveat/ptregs +322 64 execveat sys_execveat/ptregs 323 common userfaultfd sys_userfaultfd 324 common membarrier sys_membarrier 325 common mlock2 sys_mlock2
@@ -339,14 +339,14 @@ # for native 64-bit operation. # 512 x32 rt_sigaction compat_sys_rt_sigaction -513 x32 rt_sigreturn stub_x32_rt_sigreturn +513 x32 rt_sigreturn sys32_x32_rt_sigreturn 514 x32 ioctl compat_sys_ioctl 515 x32 readv compat_sys_readv 516 x32 writev compat_sys_writev 517 x32 recvfrom compat_sys_recvfrom 518 x32 sendmsg compat_sys_sendmsg 519 x32 recvmsg compat_sys_recvmsg -520 x32 execve stub_x32_execve/ptregs +520 x32 execve compat_sys_execve/ptregs 521 x32 ptrace compat_sys_ptrace 522 x32 rt_sigpending compat_sys_rt_sigpending 523 x32 rt_sigtimedwait compat_sys_rt_sigtimedwait
@@ -371,4 +371,4 @@ 542 x32 getsockopt compat_sys_getsockopt 543 x32 io_setup compat_sys_io_setup 544 x32 io_submit compat_sys_io_submit -545 x32 execveat stub_x32_execveat/ptregs +545 x32 execveat compat_sys_execveat/ptregs
Andy Lutomirski
maintainerPATCH v2 9/10 x86/entry/64: Stop using int_ret_from_sys_call in ret_from_fork
In reply to Andy Lutomirskiret_from_fork is now open-coded and is no longer tangled up with the syscall code. This isn't so bad -- this adds very little code, and IMO the result is much easier to understand.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/entry/entry_64.S | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-)
arch/x86/entry/entry_64.S +19 -16
@@ -390,7 +390,6 @@ END(ptregs_\func) * rdi: prev task we switched from */ ENTRY(ret_from_fork) - LOCK ; btr $TIF_FORK, TI_flags(%r8) pushq $0x0002
@@ -398,28 +397,32 @@ ENTRY(ret_from_fork) call schedule_tail /* rdi: 'prev' task parameter */ - RESTORE_EXTRA_REGS - testb $3, CS(%rsp) /* from kernel_thread? */ + jnz 1f /* - * By the time we get here, we have no idea whether our pt_regs, - * ti flags, and ti status came from the 64-bit SYSCALL fast path, - * the slow path, or one of the 32-bit compat paths. - * Use IRET code path to return, since it can safely handle - * all of the above. + * We came from kernel_thread. This code path is quite twisted, and + * someone should clean it up. + * + * copy_thread_tls stashes the function pointer in RBX and the + * parameter to be passed in RBP. The called function is permitted + * to call do_execve and thereby jump to user mode. */ - jnz int_ret_from_sys_call + movq RBP(%rsp), %rdi + call *RBX(%rsp) + movl $0, RAX(%rsp) /* - * We came from kernel_thread - * nb: we depend on RESTORE_EXTRA_REGS above + * Fall through as though we're exiting a syscall. This makes a + * twisted sort of sense if we just called do_execve. */ - movq %rbp, %rdi - call *%rbx - movl $0, RAX(%rsp) - RESTORE_EXTRA_REGS - jmp int_ret_from_sys_call + +1: + movq %rsp, %rdi + call syscall_return_slowpath /* returns with IRQs disabled */ + TRACE_IRQS_ON /* user mode is traced as IRQS on */ + SWAPGS + jmp restore_regs_and_iret END(ret_from_fork) /*
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] x86/entry/64: Stop using int_ret_from_sys_call in ret_from_fork
x86/entry/64: Stop using int_ret_from_sys_call in ret_from_fork
Commit-ID: 24d978b76ffd20ecff8a8d1c21b16fe740f8b119 Gitweb: http://git.kernel.org/tip/24d978b76ffd20ecff8a8d1c21b16fe740f8b119 Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:27 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:38 +0100
x86/entry/64: Stop using int_ret_from_sys_call in ret_from_fork
ret_from_fork is now open-coded and is no longer tangled up with the syscall code. This isn't so bad -- this adds very little code, and IMO the result is much easier to understand.
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/a0747e2a5e47084655a1e96351c545b755c41fa7.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/entry/entry_64.S | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-)
arch/x86/entry/entry_64.S +19 -16
@@ -390,7 +390,6 @@ END(ptregs_\func) * rdi: prev task we switched from */ ENTRY(ret_from_fork) - LOCK ; btr $TIF_FORK, TI_flags(%r8) pushq $0x0002
@@ -398,28 +397,32 @@ ENTRY(ret_from_fork) call schedule_tail /* rdi: 'prev' task parameter */ - RESTORE_EXTRA_REGS - testb $3, CS(%rsp) /* from kernel_thread? */ + jnz 1f /* - * By the time we get here, we have no idea whether our pt_regs, - * ti flags, and ti status came from the 64-bit SYSCALL fast path, - * the slow path, or one of the 32-bit compat paths. - * Use IRET code path to return, since it can safely handle - * all of the above. + * We came from kernel_thread. This code path is quite twisted, and + * someone should clean it up. + * + * copy_thread_tls stashes the function pointer in RBX and the + * parameter to be passed in RBP. The called function is permitted + * to call do_execve and thereby jump to user mode. */ - jnz int_ret_from_sys_call + movq RBP(%rsp), %rdi + call *RBX(%rsp) + movl $0, RAX(%rsp) /* - * We came from kernel_thread - * nb: we depend on RESTORE_EXTRA_REGS above + * Fall through as though we're exiting a syscall. This makes a + * twisted sort of sense if we just called do_execve. */ - movq %rbp, %rdi - call *%rbx - movl $0, RAX(%rsp) - RESTORE_EXTRA_REGS - jmp int_ret_from_sys_call + +1: + movq %rsp, %rdi + call syscall_return_slowpath /* returns with IRQs disabled */ + TRACE_IRQS_ON /* user mode is traced as IRQS on */ + SWAPGS + jmp restore_regs_and_iret END(ret_from_fork) /*
Andy Lutomirski
maintainerPATCH v2 10/10 x86/entry/64: Migrate the 64-bit syscall slow path to C
In reply to Andy LutomirskiThis is more complicated than the 32-bit and compat cases because it preserves an asm fast path for the case where the callee-saved regs aren't needed in pt_regs and no entry or exit work needs to be done.
This appears to slow down fastpath syscalls by no more than one cycle on my Skylake laptop.
Signed-off-by Andy Lutomirski <[email protected]>
--- arch/x86/entry/common.c | 26 +++++++++++ arch/x86/entry/entry_64.S | 117 ++++++++++++++++------------------------------ 2 files changed, 65 insertions(+), 78 deletions(-)
arch/x86/entry/common.c +26 -0
@@ -344,6 +344,32 @@ __visible inline void syscall_return_slowpath(struct pt_regs *regs)prepare_exit_to_usermode (regs); } +#ifdef CONFIG_X86_64 +__visible void do_syscall_64(struct pt_regs *regs) +{ +struct thread_info *ti =pt_regs_to_thread_info (regs); +unsigned long nr = regs->orig_ax; + +local_irq_enable (); + +if (READ_ONCE (ti->flags) & _TIF_WORK_SYSCALL_ENTRY) + nr =syscall_trace_enter (regs); + +/* +* NB: Native and x32 syscalls are dispatched from the same +* table. The only functional difference is the x32 bit in +* regs->orig_ax, which changes the behavior of some syscalls. +*/ +if (likely ((nr & __SYSCALL_MASK) < NR_syscalls)) { + regs->ax = sys_call_table[nr & __SYSCALL_MASK]( + regs->di, regs->si, regs->dx, + regs->r10, regs->r8, regs->r9); + } + +syscall_return_slowpath (regs); +} +#endif +#if defined(CONFIG_X86_32 ) || defined(CONFIG_IA32_EMULATION )/* * Does a 32-bit syscall. Called with IRQs on and does all entry and
arch/x86/entry/entry_64.S +39 -78
160 lines not on this page · raw message
Signature · 1 line
2.5.0
tip-bot for Andy Lutomirski
merge bot[tip:x86/asm] x86/entry/64: Migrate the 64-bit syscall slow path to C
x86/entry/64: Migrate the 64-bit syscall slow path to C
Commit-ID: 1e423bff959e48166f5b7efca01fdb0dbdf05846 Gitweb: http://git.kernel.org/tip/1e423bff959e48166f5b7efca01fdb0dbdf05846 Author: Andy Lutomirski <[email protected]> AuthorDate: Thu, 28 Jan 2016 15:11:28 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 29 Jan 2016 09:46:38 +0100
x86/entry/64: Migrate the 64-bit syscall slow path to C
This is more complicated than the 32-bit and compat cases because it preserves an asm fast path for the case where the callee-saved regs aren't needed in pt_regs and no entry or exit work needs to be done.
This appears to slow down fastpath syscalls by no more than one cycle on my Skylake laptop.
Signed-off-by Andy Lutomirski <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Brian Gerst <[email protected]>
Cc Denys Vlasenko <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
Link http://lkml.kernel.org/r/ce2335a4d42dc164b24132ee5e8c7716061f947b.1454022279.git.luto@kernel.org open here
Signed-off-by Ingo Molnar <[email protected]>
--- arch/x86/entry/common.c | 26 +++++++++++ arch/x86/entry/entry_64.S | 117 ++++++++++++++++------------------------------ 2 files changed, 65 insertions(+), 78 deletions(-)
arch/x86/entry/common.c +26 -0
@@ -344,6 +344,32 @@ __visible inline void syscall_return_slowpath(struct pt_regs *regs)prepare_exit_to_usermode (regs); } +#ifdef CONFIG_X86_64 +__visible void do_syscall_64(struct pt_regs *regs) +{ +struct thread_info *ti =pt_regs_to_thread_info (regs); +unsigned long nr = regs->orig_ax; + +local_irq_enable (); + +if (READ_ONCE (ti->flags) & _TIF_WORK_SYSCALL_ENTRY) + nr =syscall_trace_enter (regs); + +/* +* NB: Native and x32 syscalls are dispatched from the same +* table. The only functional difference is the x32 bit in +* regs->orig_ax, which changes the behavior of some syscalls. +*/ +if (likely ((nr & __SYSCALL_MASK) < NR_syscalls)) { + regs->ax = sys_call_table[nr & __SYSCALL_MASK]( + regs->di, regs->si, regs->dx, + regs->r10, regs->r8, regs->r9); + } + +syscall_return_slowpath (regs); +} +#endif +#if defined(CONFIG_X86_32 ) || defined(CONFIG_IA32_EMULATION )/* * Does a 32-bit syscall. Called with IRQs on and does all entry and
arch/x86/entry/entry_64.S +39 -78
160 lines not on this page · raw message