TIF_NOTIFY_SIGNAL for all archs
Hi Linus,
This pull sits on top of of the core entry/exit and x86 entry branch from the tip tree, which contains the generic and x86 parts of this work. Here we convert the rest of the archs to support TIF_NOTIFY_SIGNAL. With that done, we can get rid of JOBCTL_TASK_WORK from task_work and signal.c, and also remove a deadlock work-around in io_uring around knowing that signal based task_work waking is invoked with the sighand wait queue head lock.
The motivation for this work is to decouple signal notify based task_work, of which io_uring is a heavy user of, from sighand. The sighand lock becomes a huge contention point, particularly for threaded workloads where it's shared between threads. Even outside of threaded applications it's slower than it needs to be.
Roman Gershman <[email protected]> reported that his networked workload dropped from 1.6M QPS at 80% CPU to 1.0M QPS at 100% CPU after io_uring was changed to use TIF_NOTIFY_SIGNAL. The time was all spent hammering on the sighand lock, showing 57% of the CPU time there [1].
There are further cleanups possible on top of this. One example is TIF_PATCH_PENDING, where a patch already exists to use TIF_NOTIFY_SIGNAL instead. Hopefully this will also lead to more consolidation, but the work stands on its own as well.
This throws a trivial merge conflict with arm64 since you pulled that, which is just a renumbering exercise and fiddling with _TIF_WORK_MASK in that same file (arch/arm64/include/asm/thread_info.h).
[1] https://github.com/axboe/liburing/issues/215
Please pull!
The following changes since commit f8394f232b1eab649ce2df5c5f15b0e528c92091:
Linux 5.10-rc3 (2020-11-08 16:10:16 -0800)
are available in the Git repository at:
git://git.kernel.dk/linux-block.git tags/tif-task_work.arch-2020-12-14
for you to fetch changes up to 355fb9e2b78e78b38ec00f5cd9b05c6aceb98335:
io_uring: remove 'twa_signal_ok' deadlock work-around (2020-12-12 09:17:38 -0700)
----------------------------------------------------------------
tif-task_work.arch-2020-12-14
----------------------------------------------------------------
Jens Axboe (30):
Merge tag 'core-entry-notify-signal' of git://git.kernel.org/.../tip/tip into tif-task_work.arch Merge branch 'x86/entry' of git://git.kernel.org/.../tip/tip into tif-task_work.arch
arc: add support for TIF_NOTIFY_SIGNAL
arm64: add support for TIF_NOTIFY_SIGNAL
m68k: add support for TIF_NOTIFY_SIGNAL
nios32: add support for TIF_NOTIFY_SIGNAL
parisc: add support for TIF_NOTIFY_SIGNAL
powerpc: add support for TIF_NOTIFY_SIGNAL
mips: add support for TIF_NOTIFY_SIGNAL
s390: add support for TIF_NOTIFY_SIGNAL
um: add support for TIF_NOTIFY_SIGNAL
sh: add support for TIF_NOTIFY_SIGNAL
openrisc: add support for TIF_NOTIFY_SIGNAL
csky: add support for TIF_NOTIFY_SIGNAL
hexagon: add support for TIF_NOTIFY_SIGNAL
microblaze: add support for TIF_NOTIFY_SIGNAL
arm: add support for TIF_NOTIFY_SIGNAL
xtensa: add support for TIF_NOTIFY_SIGNAL
alpha: add support for TIF_NOTIFY_SIGNAL
c6x: add support for TIF_NOTIFY_SIGNAL
h8300: add support for TIF_NOTIFY_SIGNAL
ia64: add support for TIF_NOTIFY_SIGNAL
nds32: add support for TIF_NOTIFY_SIGNAL
riscv: add support for TIF_NOTIFY_SIGNAL
sparc: add support for TIF_NOTIFY_SIGNAL
task_work: remove legacy TWA_SIGNAL path
io_uring: JOBCTL_TASK_WORK is no longer used by task_work
signal: kill JOBCTL_TASK_WORK
kernel: remove checking for TIF_NOTIFY_SIGNAL
io_uring: remove 'twa_signal_ok' deadlock work-around
arch/alpha/include/asm/thread_info.h | 2 ++ arch/alpha/kernel/entry.S | 2 +- arch/alpha/kernel/signal.c | 2 +- arch/arc/include/asm/thread_info.h | 4 ++- arch/arc/kernel/entry.S | 3 ++- arch/arc/kernel/signal.c | 2 +- arch/arm/include/asm/thread_info.h | 7 +++++- arch/arm/kernel/entry-common.S | 6 ++--- arch/arm/kernel/entry-v7m.S | 2 +- arch/arm/kernel/signal.c | 2 +- arch/arm64/include/asm/thread_info.h | 5 +++- arch/arm64/kernel/signal.c | 2 +- arch/c6x/include/asm/thread_info.h | 1 + arch/c6x/kernel/asm-offsets.c | 1 + arch/c6x/kernel/signal.c | 3 ++- arch/csky/include/asm/thread_info.h | 5 +++- arch/csky/kernel/signal.c | 2 +- arch/h8300/include/asm/thread_info.h | 4 ++- arch/h8300/kernel/signal.c | 2 +- arch/hexagon/include/asm/thread_info.h | 2 ++ arch/hexagon/kernel/process.c | 2 +- arch/ia64/include/asm/thread_info.h | 4 ++- arch/ia64/kernel/process.c | 3 ++- arch/m68k/include/asm/thread_info.h | 1 + arch/m68k/kernel/signal.c | 3 ++- arch/microblaze/include/asm/thread_info.h | 2 ++ arch/microblaze/kernel/signal.c | 3 ++- arch/mips/include/asm/thread_info.h | 4 ++- arch/mips/kernel/signal.c | 2 +- arch/nds32/include/asm/thread_info.h | 2 ++ arch/nds32/kernel/ex-exit.S | 2 +- arch/nds32/kernel/signal.c | 2 +- arch/nios2/include/asm/thread_info.h | 2 ++ arch/nios2/kernel/signal.c | 3 ++- arch/openrisc/include/asm/thread_info.h | 2 ++ arch/openrisc/kernel/signal.c | 2 +- arch/parisc/include/asm/thread_info.h | 4 ++- arch/parisc/kernel/signal.c | 3 ++- arch/powerpc/include/asm/thread_info.h | 5 +++- arch/powerpc/kernel/signal.c | 2 +- arch/riscv/include/asm/thread_info.h | 5 +++- arch/riscv/kernel/signal.c | 2 +- arch/s390/include/asm/thread_info.h | 2 ++ arch/s390/kernel/entry.S | 11 +++++---- arch/s390/kernel/signal.c | 2 +- arch/sh/include/asm/thread_info.h | 4 ++- arch/sh/kernel/signal_32.c | 2 +- arch/sparc/include/asm/thread_info_32.h | 4 ++- arch/sparc/include/asm/thread_info_64.h | 6 +++-- arch/sparc/kernel/signal_32.c | 2 +- arch/sparc/kernel/signal_64.c | 2 +- arch/um/include/asm/thread_info.h | 2 ++ arch/um/kernel/process.c | 3 ++- arch/xtensa/include/asm/thread_info.h | 5 +++- arch/xtensa/kernel/entry.S | 4 +-- arch/xtensa/kernel/signal.c | 3 ++- fs/io_uring.c | 30 ++++++----------------- include/linux/entry-common.h | 4 --- include/linux/sched/jobctl.h | 4 +-- include/linux/sched/signal.h | 2 -- include/linux/tracehook.h | 4 --- kernel/signal.c | 22 ----------------- kernel/task_work.c | 30 +---------------------- 63 files changed, 128 insertions(+), 139 deletions(-)
Signature · 1 line
Jens Axboe
The pull request you sent on Wed, 16 Dec 2020 07:33:42 -0700:
The pull request you sent on Wed, 16 Dec 2020 07:33:42 -0700:
Quoted 1 line
git://git.kernel.dk/linux-block.gittags/tif-task_work.arch-2020-12-14
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/005b2a9dc819a1265a8c765595f8f6d88d6173d9
Thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/prtracker.html
Mark Rutland
reviewerPATCH 1/1 arm64: make _TIF_WORK_MASK bits contiguous (was: Re: [GIT PULL] TIF_NOTIFY_SIGNAL for all archs)
Hi Linus,
Quoted 4 lines · On Wed, Dec 16, 2020 at 09:17:39PM +0000, [email protected] wrote:
Quoted 1 line
git://git.kernel.dk/linux-block.gittags/tif-task_work.arch-2020-12-14has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/005b2a9dc819a1265a8c765595f8f6d88d6173d9
Unfortunately the merge resolution broke the build for arm64 -- could you please apply the fixup below? IIUC that matches what we did in linux-next, and builds fine locally.
Thanks,
Mark.
---->8----
From da1826cc5132bb9d46aebdb47fa033f94825a697 Mon Sep 17 00:00:00 2001
From: Mark Rutland <[email protected]>
Date: Wed, 16 Dec 2020 21:32:48 +0000
Subject: [PATCH] arm64: make _TIF_WORK_MASK bits contiguous
We need the bits of _TIF_WORK_MASK to be contiguous in order to to use this as an immediate argument to an AND instruction in entry.S.
We happened to change these bits in commits:
b5a5a01d8e9a44ec ("arm64: uaccess: remove addr_limit_user_check()") 192caabd4dd92c98 ("arm64: add support for TIF_NOTIFY_SIGNAL")
... which each worked in isolation, but the merge resolution in commit:
005b2a9dc819a126 ("Merge tag 'tif-task_work.arch-2020-12-14' of git://git.kernel.dk/linux-block")
... happened to make the bits non-contiguous.
Fix this by moving TIF_NOTIFY_SIGNAL to be bit 6, which is contiguous with the rest of _TIF_WORK_MASK.
Otherwise, we'll get a build-time failure as below:
| arch/arm64/kernel/entry.S: Assembler messages: | arch/arm64/kernel/entry.S:733: Error: immediate out of range at operand 3 -- `and x2,x19,#((1<<1)|(1<<0)|(1<<2)|(1<<3)|(1<<4)|(1<<5)|(1<<7))' | scripts/Makefile.build:360: recipe for target 'arch/arm64/kernel/entry.o' failed | make[2]: *** [arch/arm64/kernel/entry.o] Error 1 | scripts/Makefile.build:496: recipe for target 'arch/arm64/kernel' failed | make[1]: *** [arch/arm64/kernel] Error 2 | Makefile:1805: recipe for target 'arch/arm64' failed | make: *** [arch/arm64] Error 2
--- arch/arm64/include/asm/thread_info.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/include/asm/thread_info.h +1 -1
@@ -64,7 +64,7 @@ void arch_release_task_struct(struct task_struct *tsk);#define TIF_FOREIGN_FPSTATE 3/* CPU's FP state is not current's */ #define TIF_UPROBE 4/* uprobe breakpoint or singlestep */ #define TIF_MTE_ASYNC_FAULT 5/* MTE Asynchronous Tag Check Fault */ -#define TIF_NOTIFY_SIGNAL 7/* signal notifications exist */ +#define TIF_NOTIFY_SIGNAL 6/* signal notifications exist */ #define TIF_SYSCALL_TRACE 8/* syscall trace active */ #define TIF_SYSCALL_AUDIT 9/* syscall auditing */ #define TIF_SYSCALL_TRACEPOINT 10/* syscall tracepoint for ftrace */
Signature · 1 line
2.11.0
Re: [PATCH] arm64: make _TIF_WORK_MASK bits contiguous (was: Re: [GIT PULL] TIF_NOTIFY_SIGNAL for all archs)
Quoted 4 lines · Mark Rutland
Unfortunately the merge resolution broke the build for arm64 -- could you please apply the fixup below? IIUC that matches what we did in linux-next, and builds fine locally.
Oops. That was a bit too subtle. I didn't realize that the bits really wanted to be contiguous, and it wasn't obvious at the point of conflict.
Will apply your fix asap.
Linus
Re: [PATCH] arm64: make _TIF_WORK_MASK bits contiguous
Quoted 11 lines · Linus Torvalds
Quoted 4 lines · Mark Rutland
Unfortunately the merge resolution broke the build for arm64 -- could you please apply the fixup below? IIUC that matches what we did in linux-next, and builds fine locally.
Oops. That was a bit too subtle. I didn't realize that the bits really wanted to be contiguous, and it wasn't obvious at the point of conflict.
Will apply your fix asap.
Thanks (both Mark and Linus), I didn't know they had to be contiguous either as an immediate value...
Signature · 1 line
Jens Axboe
Catalin Marinas
maintainerRe: [PATCH] arm64: make _TIF_WORK_MASK bits contiguous (was: Re: [GIT PULL] TIF_NOTIFY_SIGNAL for all archs)
In reply to Linus TorvaldsQuoted 10 lines · Linus Torvalds
Quoted 3 lines · Mark Rutland
Unfortunately the merge resolution broke the build for arm64 -- could you please apply the fixup below? IIUC that matches what we did in linux-next, and builds fine locally.
Oops. That was a bit too subtle. I didn't realize that the bits really wanted to be contiguous, and it wasn't obvious at the point of conflict.
Will apply your fix asap.
Thanks Linus for applying it and thanks Mark for the quick fix. I noticed it in -next before and gave Stephen my conflict resolution.
Signature · 1 line
Catalin