arm64: entry: Convert to Generic Entry

This series converts arm64 to the generic entry infrastructure.

Currently, architectures such as x86, RISC-V, LoongArch, PowerPC and S390 use generic entry, which significantly reduces maintenance burden and improves code elegance. arm64 already adopted generic IRQ entry in commit b3cf07851b6c ("arm64: entry: Switch to generic IRQ entry"), making it the right time to complete the conversion.

This work brings arm64 in line with other major architectures, reducing duplicated code and enabling future improvements like Syscall User Dispatch and rseq time slice extension optimizations.

What this series does
=====================

Patch 1-2: Fix two issues related to rseq and pseudo single-step. Patch 3-12: Do some minor refactoring to gradually align the generic
entry code, without functional changes intended.
Patch 13: Officially switch ARM64 to the generic entry, used the
functions provided by the framework, and deleted the duplicate code corresponding to ARM64.
Patch 14: Inline el0_svc_common() function to prevent performance
regression.
Patch 15: Add related ptrace test cases and rseq test cases for two
bug fixes.

Testing
========

Observable (non-ABI) changes worth calling out:

- The traceable functions on the syscall path change: syscall_trace_enter, syscall_trace_exit and el0_svc_common.constprop.0 are replaced by trace_syscall_enter, trace_syscall_exit and syscall_enter_audit, the same names x86, RISC-V, LoongArch already expose.

- CONFIG_DEBUG_RSEQ becomes unselectable on arm64 once GENERIC_ENTRY is set (rseq_syscall() becomes a no-op stub). The equivalent check is done by __rseq_debug_syscall_return(), enabled with the "rseq_debug=" boot option.

The new added selftests pass, and the rseq test also passes with "rseq_debug=1" after the conversion.

On a KVM VM (2 vCPU pinned, LOCKDEP off), with the same config, the syscall hot path shows no measurable regression. Each row below is the mean of 10 runs:

    | Benchmark                         | baseline | generic entry |
    | --------------------------------- | -------- | ------------- |
    | UnixBench System Call Overhead    | 879982   | 877908        |
    | perf bench syscall basic 1M (ns)  | 197.29   | 197.37        |

Both differences are within the noise, so there is no measurable impact on syscall overhead.

Stress-ng (syscall and cpu) and the kselftests (breakpoints, arm64/abi, arm64/fp, vDSO) were also run and all pass.

The hackbench, ptrace stress, Pseudo-NMI and rr test suite [1] reported for v20 were not repeated as the code unchanged apart from the rebase.

On gVisor with systrap mode, the Syscall User Dispatch (SUD) feature yields a slightly lower performance gain on arm64 (3% ~ 5%), similar to that on x86 [2].

Result: the average over 1M getpid calls, ns/op

    | KVM VM Configuration | SUD    | no-SUD | Savings % |
    | -------------------  | -----  | -------| --------- |
    | 2 vCPU, pinned       | 4269.8 | 4509.5 | 5.3%      |
    | 4 vCPU, pinned       | 4175.4 | 4369.1 | 4.4%      |
    | 8 vCPU, pinned       | 4352.6 | 4485.4 | 3.0%      |

Note that this series does not yet enable SUD or the rseq time slice extension on arm64: SUD additionally needs ARCH_SUPPORTS_SYSCALL_USER_DISPATCH [3] and the rseq slice extension needs HAVE_GENERIC_TIF_BITS [4], neither of which arm64 selects today. The numbers above are the motivation for enabling them in a follow-up.

The new selftests and tests are helped by an agent.

Testing follow-up that still requires real hardware:

- MTE: run the MTE selftests on real hardware; part of them do not run under QEMU.

[1]: https://github.com/rr-debugger/rr/wiki/Building-And-Installing#tests
[2]: https://github.com/google/gvisor/commit/a45d6227b521012109cde9a942d644d778eb773b
[3]: https://lore.kernel.org/all/[email protected]/
[4]: https://lore.kernel.org/all/[email protected]/

The series is rebased on v7.4-rc1 and also available at:
https://github.com/ruanjinjie-eng/linux/commits/arm64_generic_entry_v21

Changes in v21:
- Rebased on v7.4-rc1.
- Pseudo-single-step patch rewritten as a bugfix.
- Rseq patch rewritten as a bugfix and moved to the front.
- Retitle and update the commit message.
- Selftests: replace the 2 SYSEMU tests with 5 tests that reproduce
the problems fixed by the commit ac2081cdc4d99 ("arm64: ptrace:
Consistently use pseudo-singlestep exceptions") series and the rseq issue.
- Solve the sashiko commit message issue.
- Link to v20: https://lore.kernel.org/all/[email protected]/

Changes in v20:
- Add a kselftest covering the syscall-exit pseudo-step
and single-step behaviour.
- Reword the first patch: it skips the pseudo-step for the emulated
syscall rather than "fixing a redundant exit stop", and drop the
Fixes: tag and Reviewed-by.
- Add Tested-by.
- Cover letter: document the observable changes and the testing
follow-ups.
- Link to v19: https://lore.kernel.org/all/[email protected]/

Changes in v19:
- Rebased on v7.3-rc4.
- Fix build problem as Karl pointed out.
- Remove has_syscall_work() later to make it more clear.
- Add gvisor test data.
- Link to v18: https://lore.kernel.org/all/[email protected]/

For changes in v7 to v17, please refer to the individual patch threads:
- v17: https://lore.kernel.org/all/[email protected]/
- v16: https://lore.kernel.org/all/[email protected]/
- v15: https://lore.kernel.org/all/[email protected]/
- v14: https://lore.kernel.org/all/[email protected]/
- v13: https://lore.kernel.org/all/[email protected]/
- v12: https://lore.kernel.org/all/[email protected]/
- v11: https://lore.kernel.org/all/[email protected]/
- v10: https://lore.kernel.org/all/[email protected]/
- v9: https://lore.kernel.org/all/[email protected]/
- v8: https://lore.kernel.org/all/[email protected]/
- v7: https://lore.kernel.org/all/[email protected]/

Jinjie Ruan (15):
arm64: ptrace: Fix spurious pseudo-single-step for
PTRACE_SYSEMU_SINGLESTEP
arm64: ptrace: Check rseq critical section before reporting syscall
exit
arm64: ptrace: Add syscall_enter_audit() helper
arm64: ptrace: Open-code seccomp check in syscall_trace_enter()
arm64: ptrace: Rename and clean up syscall_trace_enter()
arm64: syscall: Invert the exit-path flags check in el0_svc_common()
arm64: ptrace: Pass thread flags to trace enter/exit
arm64: ptrace: Extract arm64_syscall_exit_to_user_mode_work() helper
arm64: ptrace: Only run syscall exit work when exit work flags are set
arm64: syscall: Use exit-specific flags check in el0_svc_common()
arm64: syscall: Simplify el0_svc_common() syscall exit path
arm64: ptrace: Return bool from arm64_syscall_trace_enter()
arm64: entry: Convert to generic entry
arm64: Inline el0_svc_common()
selftests/arm64: Add ptrace single-step and rseq syscall-exit tests

 arch/arm64/Kconfig                            |   2 +-
 arch/arm64/include/asm/entry-common.h         |  77 ++++++++
 arch/arm64/include/asm/syscall.h              |   5 +-
 arch/arm64/include/asm/thread_info.h          |  16 +-
 arch/arm64/kernel/debug-monitors.c            |   8 +
 arch/arm64/kernel/entry-common.c              |   2 +-
 arch/arm64/kernel/ptrace.c                    | 115 ------------
 arch/arm64/kernel/signal.c                    |   2 +-
 arch/arm64/kernel/syscall.c                   |  34 ++--
 tools/testing/selftests/arm64/Makefile        |   2 +-
 tools/testing/selftests/arm64/ptrace/Makefile |  12 ++
 .../selftests/arm64/ptrace/ptrace_common.h    | 101 ++++++++++
 .../selftests/arm64/ptrace/rseq_syscall_pc.c  | 130 +++++++++++++
 .../selftests/arm64/ptrace/rseq_syscall_pc.h  |  22 +++
 .../arm64/ptrace/rseq_syscall_pc_asm.S        |  45 +++++
 .../arm64/ptrace/single_step_signal_handler.c | 121 ++++++++++++
 .../arm64/ptrace/single_step_spsr_ss.c        |  79 ++++++++
 .../arm64/ptrace/single_step_syscall_stop.c   | 138 ++++++++++++++
 .../arm64/ptrace/sysemu_singlestep.c          | 172 ++++++++++++++++++
 19 files changed, 923 insertions(+), 160 deletions(-)
 create mode 100644 tools/testing/selftests/arm64/ptrace/Makefile
 create mode 100644 tools/testing/selftests/arm64/ptrace/ptrace_common.h
 create mode 100644 tools/testing/selftests/arm64/ptrace/rseq_syscall_pc.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/rseq_syscall_pc.h
 create mode 100644 tools/testing/selftests/arm64/ptrace/rseq_syscall_pc_asm.S
 create mode 100644 tools/testing/selftests/arm64/ptrace/single_step_signal_handler.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/single_step_spsr_ss.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/single_step_syscall_stop.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/sysemu_singlestep.c

2.34.1

PTRACE_SYSEMU_SINGLESTEP sets both _TIF_SYSCALL_EMU and _TIF_SINGLESTEP. Since commit ac2081cdc4d99 ("arm64: ptrace: Consistently use pseudo-singlestep exceptions"), syscall_trace_exit() injects a pseudo-single-step SIGTRAP (si_code SI_USER) whenever _TIF_SINGLESTEP is set, including for an emulated system call.

That pseudo-step was added for PTRACE_SINGLESTEP: the system call runs, and the tracer must get a stop at the return address to inspect the result. An emulated system call does not run, and its syscall-enter-stop already reports the SVC, as the x86 implementation and generic entry (report_single_step()) both say. The extra stop is wrong:

1) Stepping armed before the SVC (the normal case). The SVC exception leaves the step pending (SPSR_EL1.SS is 0, Arm ARM D2.11.5.1), so the hardware step is taken at the return address anyway, and the tracer now stops there twice without the tracee executing anything:

	syscall-enter-stop  pc=L
	pseudo-step         pc=L    si_code=SI_USER  <- added by that commit
	hardware step       pc=L    si_code=TRAP_TRACE
	hardware step       pc=L+4

2) Stepping armed after the SVC, e.g. a tracer resuming a PTRACE_EVENT_SECCOMP stop with PTRACE_SYSEMU_SINGLESTEP. The first stop is the pseudo-step at L, so the instruction at L is not executed before the step is reported. Before that commit, and on x86, the first stop follows the instruction at L.

Skip the pseudo-step when _TIF_SYSCALL_EMU is set, as generic entry does. The behaviour of PTRACE_SYSEMU_SINGLESTEP goes back to what it was before that commit. PTRACE_SINGLESTEP is unaffected and still gets the pseudo-step, so both problems fixed by that commit stay fixed.

Only tracers using PTRACE_SYSEMU_SINGLESTEP see the difference. Measured in QEMU, L is the instruction after the SVC:

  case          before that commit   v7.4-rc1      this patch
  normal        L, L+4               L, L, L+4     L, L+4
  after seccomp L+4                  L             L+4

---
 arch/arm64/kernel/ptrace.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

index 7405c76724bf..4baf04f20e67 100644
@@ -2482,16 +2482,26 @@ int syscall_trace_enter(struct pt_regs *regs)
 	return regs->syscallno;
 }
 
+static inline bool report_single_step(unsigned long flags)
+{
+	if (flags & _TIF_SYSCALL_EMU)
+		return false;
+
+	return flags & _TIF_SINGLESTEP;
+}
+
 void syscall_trace_exit(struct pt_regs *regs)
 {
 	unsigned long flags = read_thread_flags();
+	bool step;
 
 	audit_syscall_exit(regs);
 
 	if (flags & _TIF_SYSCALL_TRACEPOINT)
 		trace_sys_exit(regs, syscall_get_return_value(current, regs));
 
-	if (flags & (_TIF_SYSCALL_TRACE | _TIF_SINGLESTEP))
+	step = report_single_step(flags);
+	if (step || flags & _TIF_SYSCALL_TRACE)
 		report_syscall_exit(regs);
 
 	rseq_syscall(regs);

2.34.1

With CONFIG_DEBUG_RSEQ=y, rseq_syscall() kills the task with SIGSEGV if the system call was issued from inside an rseq critical section. It decides this from regs->pc.

arm64 calls rseq_syscall() after report_syscall_exit(), so a tracer stopped at the syscall-exit-stop can change regs->pc with PTRACE_SETREGSET before the check runs. The check then looks at the PC written by the tracer, not the PC the system call was issued from:

- a system call issued inside a critical section is not detected if the tracer moves the PC out of it;

- a system call issued outside a critical section kills the task with SIGSEGV if the tracer moves the PC into it.

Call rseq_syscall() first, before audit, tracepoints and ptrace, as generic entry (rseq_debug_syscall_return()), x86 before it moved to generic entry, and arm32 already do.

Only kernels with CONFIG_DEBUG_RSEQ=y are affected. The option depends on EXPERT and DEBUG_KERNEL, is off by default, and is otherwise a no-op stub. Measured in QEMU with a traced task that issues getppid():

  case                                  before    after
  inside the CS, PC moved out           not killed SIGSEGV
  outside the CS, PC moved into the CS  SIGSEGV    exits normally

---
 arch/arm64/kernel/ptrace.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 4baf04f20e67..a34d4893f4bf 100644
@@ -2495,6 +2495,8 @@ void syscall_trace_exit(struct pt_regs *regs)
 	unsigned long flags = read_thread_flags();
 	bool step;
 
+	rseq_syscall(regs);
+
 	audit_syscall_exit(regs);
 
 	if (flags & _TIF_SYSCALL_TRACEPOINT)

@@ -2503,8 +2505,6 @@ void syscall_trace_exit(struct pt_regs *regs)
 	step = report_single_step(flags);
 	if (step || flags & _TIF_SYSCALL_TRACE)
 		report_syscall_exit(regs);
-
-	rseq_syscall(regs);
 }
 
 /*

2.34.1

Extract syscall_enter_audit() from arm64_syscall_trace_enter() and make it match the generic entry version:

- Check audit_context() in the caller, as audit_syscall_entry() did, so nothing is done when audit is not active.

- Use syscall_get_nr() and syscall_get_arguments() instead of reading regs->syscallno and the regs fields directly.

This is a preparation step for converting arm64 to the generic entry infrastructure. No functional changes intended.

---
 arch/arm64/kernel/ptrace.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

index a34d4893f4bf..22bda379b244 100644
@@ -2458,6 +2458,19 @@ static void report_syscall_exit(struct pt_regs *regs)
 	}
 }
 
+#ifdef CONFIG_AUDITSYSCALL
+static inline void syscall_enter_audit(struct pt_regs *regs)
+{
+	long syscall = syscall_get_nr(current, regs);
+	unsigned long args[6];
+
+	syscall_get_arguments(current, regs, args);
+	__audit_syscall_entry(syscall, args[0], args[1], args[2], args[3]);
+}
+#else
+static inline void syscall_enter_audit(struct pt_regs *regs) { }
+#endif
+
 int syscall_trace_enter(struct pt_regs *regs)
 {
 	unsigned long flags = read_thread_flags();

@@ -2476,8 +2489,8 @@ int syscall_trace_enter(struct pt_regs *regs)
 	if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
 		trace_sys_enter(regs, regs->syscallno);
 
-	audit_syscall_entry(regs->syscallno, regs->orig_x0, regs->regs[1],
-			    regs->regs[2], regs->regs[3]);
+	if (unlikely(audit_context()))
+		syscall_enter_audit(regs);
 
 	return regs->syscallno;
 }

2.34.1

Refactor syscall_trace_enter() by open-coding the seccomp check to align with the generic entry framework.

The generic entry implementation expands the seccomp check in-place by testing SYSCALL_WORK_SECCOMP and directly calling the underlying __seccomp_permit_syscall() function. Moreover, generic entry explicitly re-reads work flags after ptrace handling to ensure any updates to seccomp work flags during the ptrace stop are observed.

Bring arm64 in line with this behavior:
- Re-read thread flags after ptrace handling.

- Test the updated flags for _TIF_SECCOMP and call __seccomp_permit_syscall().

No functional changes are intended; this change simplifies future migration to the generic entry framework.

---
 arch/arm64/kernel/ptrace.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

index 22bda379b244..9c45ff796732 100644
@@ -2480,11 +2480,16 @@ int syscall_trace_enter(struct pt_regs *regs)
 		ret = report_syscall_entry(regs);
 		if (ret || (flags & _TIF_SYSCALL_EMU))
 			return NO_SYSCALL;
+
+		/* ptrace might have changed thread flags */
+		flags = read_thread_flags();
 	}
 
 	/* Do the secure computing after ptrace; failures should be fast. */
-	if (!seccomp_permit_syscall())
-		return NO_SYSCALL;
+	if (unlikely(flags & _TIF_SECCOMP)) {
+		if (!__seccomp_permit_syscall())
+			return NO_SYSCALL;
+	}
 
 	if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
 		trace_sys_enter(regs, regs->syscallno);

2.34.1

Rename syscall_trace_enter() to arm64_syscall_trace_enter() to avoid name collisions and clarify the boundary when arm64 eventually switches to the generic entry infrastructure.

In addition, replace direct accesses to regs->syscallno with the standard syscall_get_nr() helper for both trace_sys_enter() and the return value. This decouples the tracing logic from architecture-specific struct pt_regs layouts, aligning the implementation with the generic entry pattern.

No functional changes intended; this is a preparation step for converting arm64 to the generic entry infrastructure.

---
 arch/arm64/include/asm/syscall.h | 2 +-
 arch/arm64/kernel/ptrace.c       | 6 +++---
 arch/arm64/kernel/syscall.c      | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

index 5e4c7fc44f73..d1dabdcde41c 100644
@@ -120,7 +120,7 @@ static inline int syscall_get_arch(struct task_struct *task)
 	return AUDIT_ARCH_AARCH64;
 }
 
-int syscall_trace_enter(struct pt_regs *regs);
+int arm64_syscall_trace_enter(struct pt_regs *regs);
 void syscall_trace_exit(struct pt_regs *regs);
 
 #endif	/* __ASM_SYSCALL_H */

index 9c45ff796732..1d93c3e1253a 100644
@@ -2471,7 +2471,7 @@ static inline void syscall_enter_audit(struct pt_regs *regs)
 static inline void syscall_enter_audit(struct pt_regs *regs) { }
 #endif
 
-int syscall_trace_enter(struct pt_regs *regs)
+int arm64_syscall_trace_enter(struct pt_regs *regs)
 {
 	unsigned long flags = read_thread_flags();
 	int ret;

@@ -2492,12 +2492,12 @@ int syscall_trace_enter(struct pt_regs *regs)
 	}
 
 	if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
-		trace_sys_enter(regs, regs->syscallno);
+		trace_sys_enter(regs, syscall_get_nr(current, regs));
 
 	if (unlikely(audit_context()))
 		syscall_enter_audit(regs);
 
-	return regs->syscallno;
+	return syscall_get_nr(current, regs);
 }
 
 static inline bool report_single_step(unsigned long flags)

index 358ddfbf1401..3e78e159b2a1 100644
@@ -113,7 +113,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 		 */
 		if (scno == NO_SYSCALL)
 			syscall_set_return_value(current, regs, -ENOSYS, 0);
-		scno = syscall_trace_enter(regs);
+		scno = arm64_syscall_trace_enter(regs);
 		if (scno == NO_SYSCALL)
 			goto trace_exit;
 	}

2.34.1

In the fast exit path of el0_svc_common(), invert the nested flags check so that it calls syscall_trace_exit() directly when work is pending and then returns, instead of returning early when no work is pending and falling through to the trace_exit label.

This makes the fast path call the exit work in the same place as the slow path, which later patches fold into a single helper.

In preparation for moving arm64 over to the generic entry. No functional changes.

---
 arch/arm64/kernel/syscall.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

index 3e78e159b2a1..b8d7d29a431b 100644
@@ -127,8 +127,9 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 	 */
 	if (!has_syscall_work(flags) && !IS_ENABLED(CONFIG_DEBUG_RSEQ)) {
 		flags = read_thread_flags();
-		if (!has_syscall_work(flags) && !(flags & _TIF_SINGLESTEP))
-			return;
+		if (has_syscall_work(flags) || flags & _TIF_SINGLESTEP)
+			syscall_trace_exit(regs);
+		return;
 	}
 
 trace_exit:

2.34.1

Move the reading of thread flags from inside arm64_syscall_trace_enter() and syscall_trace_exit() to their callers. This aligns the function signatures with the generic entry framework, where the caller is responsible for supplying the flags.

In el0_svc_common(), the flags are now passed directly to the tracing functions, and re-read before the enter/exit path to reflect any updates.

No functional change intended; this is a preparatory step for converting arm64 to the generic entry infrastructure.

---
 arch/arm64/include/asm/syscall.h |  4 ++--
 arch/arm64/kernel/ptrace.c       |  6 ++----
 arch/arm64/kernel/syscall.c      | 11 +++++++----
 3 files changed, 11 insertions(+), 10 deletions(-)

index d1dabdcde41c..696739f5a250 100644
@@ -120,7 +120,7 @@ static inline int syscall_get_arch(struct task_struct *task)
 	return AUDIT_ARCH_AARCH64;
 }
 
-int arm64_syscall_trace_enter(struct pt_regs *regs);
-void syscall_trace_exit(struct pt_regs *regs);
+int arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags);
+void syscall_trace_exit(struct pt_regs *regs, unsigned long flags);
 
 #endif	/* __ASM_SYSCALL_H */

index 1d93c3e1253a..043f48ddc168 100644
@@ -2471,9 +2471,8 @@ static inline void syscall_enter_audit(struct pt_regs *regs)
 static inline void syscall_enter_audit(struct pt_regs *regs) { }
 #endif
 
-int arm64_syscall_trace_enter(struct pt_regs *regs)
+int arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags)
 {
-	unsigned long flags = read_thread_flags();
 	int ret;
 
 	if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) {

@@ -2508,9 +2507,8 @@ static inline bool report_single_step(unsigned long flags)
 	return flags & _TIF_SINGLESTEP;
 }
 
-void syscall_trace_exit(struct pt_regs *regs)
+void syscall_trace_exit(struct pt_regs *regs, unsigned long flags)
 {
-	unsigned long flags = read_thread_flags();
 	bool step;
 
 	rseq_syscall(regs);

index b8d7d29a431b..e778aac6fab9 100644
@@ -113,7 +113,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 		 */
 		if (scno == NO_SYSCALL)
 			syscall_set_return_value(current, regs, -ENOSYS, 0);
-		scno = arm64_syscall_trace_enter(regs);
+		scno = arm64_syscall_trace_enter(regs, read_thread_flags());
 		if (scno == NO_SYSCALL)
 			goto trace_exit;
 	}

@@ -127,13 +127,16 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 	 */
 	if (!has_syscall_work(flags) && !IS_ENABLED(CONFIG_DEBUG_RSEQ)) {
 		flags = read_thread_flags();
-		if (has_syscall_work(flags) || flags & _TIF_SINGLESTEP)
-			syscall_trace_exit(regs);
+		if (has_syscall_work(flags) || flags & _TIF_SINGLESTEP) {
+			flags = read_thread_flags();
+			syscall_trace_exit(regs, flags);
+		}
 		return;
 	}
 
 trace_exit:
-	syscall_trace_exit(regs);
+	flags = read_thread_flags();
+	syscall_trace_exit(regs, flags);
 }
 
 void do_el0_svc(struct pt_regs *regs)

2.34.1

Introduce a helper arm64_syscall_exit_to_user_mode_work() that encapsulates the re-reading of thread flags and the call to syscall_trace_exit(). Use it in el0_svc_common() to replace the open-coded instances, removing the duplicated flags read and making the control flow simpler.

This aligns the arm64 exit path with the pattern expected by the generic entry infrastructure, which uses a similar syscall_exit_to_user_mode_work() callback.

No functional changes intended; this is a preparation step for converting arm64 to the generic entry infrastructure.

---
 arch/arm64/include/asm/syscall.h | 7 +++++++
 arch/arm64/kernel/syscall.c      | 9 +++------
 2 files changed, 10 insertions(+), 6 deletions(-)

index 696739f5a250..3e4672ab4a4f 100644
@@ -123,4 +123,11 @@ static inline int syscall_get_arch(struct task_struct *task)
 int arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags);
 void syscall_trace_exit(struct pt_regs *regs, unsigned long flags);
 
+static __always_inline void arm64_syscall_exit_to_user_mode_work(struct pt_regs *regs)
+{
+	unsigned long flags = read_thread_flags();
+
+	syscall_trace_exit(regs, flags);
+}
+
 #endif	/* __ASM_SYSCALL_H */

index e778aac6fab9..0061fc63e7ba 100644
@@ -127,16 +127,13 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 	 */
 	if (!has_syscall_work(flags) && !IS_ENABLED(CONFIG_DEBUG_RSEQ)) {
 		flags = read_thread_flags();
-		if (has_syscall_work(flags) || flags & _TIF_SINGLESTEP) {
-			flags = read_thread_flags();
-			syscall_trace_exit(regs, flags);
-		}
+		if (has_syscall_work(flags) || flags & _TIF_SINGLESTEP)
+			arm64_syscall_exit_to_user_mode_work(regs);
 		return;
 	}
 
 trace_exit:
-	flags = read_thread_flags();
-	syscall_trace_exit(regs, flags);
+	arm64_syscall_exit_to_user_mode_work(regs);
 }
 
 void do_el0_svc(struct pt_regs *regs)

2.34.1

Make the syscall exit path match generic entry, where syscall_exit_work() is only called when one of the SYSCALL_WORK_EXIT bits is set:

- Add _TIF_SYSCALL_EXIT_WORK for the thread flags that need work on syscall exit: trace, audit and tracepoint.

- Call the exit work from arm64_syscall_exit_to_user_mode_work() only when one of those flags or _TIF_SINGLESTEP is set.

- Rename syscall_trace_exit() to arm64_syscall_exit_work(), the arm64 counterpart of syscall_exit_work().

- Move rseq_syscall() out of arm64_syscall_exit_work() into arm64_syscall_exit_to_user_mode_work(), so it still runs on every syscall exit and still runs before ptrace can change the PC, as in generic entry.

Checking _TIF_SYSCALL_AUDIT instead of calling audit_syscall_exit() unconditionally is equivalent: the flag is set together with the audit context at fork and neither changes while the task runs a system call.

No functional changes intended.

---
 arch/arm64/include/asm/syscall.h     | 8 ++++++--
 arch/arm64/include/asm/thread_info.h | 3 +++
 arch/arm64/kernel/ptrace.c           | 5 +----
 3 files changed, 10 insertions(+), 6 deletions(-)

index 3e4672ab4a4f..7cc872fde019 100644
@@ -8,6 +8,7 @@
 #include <uapi/linux/audit.h>
 #include <linux/compat.h>
 #include <linux/err.h>
+#include <linux/rseq.h>
 
 typedef long (*syscall_fn_t)(const struct pt_regs *regs);
 

@@ -121,13 +122,16 @@ static inline int syscall_get_arch(struct task_struct *task)
 }
 
 int arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags);
-void syscall_trace_exit(struct pt_regs *regs, unsigned long flags);
+void arm64_syscall_exit_work(struct pt_regs *regs, unsigned long flags);
 
 static __always_inline void arm64_syscall_exit_to_user_mode_work(struct pt_regs *regs)
 {
 	unsigned long flags = read_thread_flags();
 
-	syscall_trace_exit(regs, flags);
+	rseq_syscall(regs);
+
+	if (unlikely(flags & _TIF_SYSCALL_EXIT_WORK) || flags & _TIF_SINGLESTEP)
+		arm64_syscall_exit_work(regs, flags);
 }
 
 #endif	/* __ASM_SYSCALL_H */

index 6cb976c1c693..a85c72b65b82 100644
@@ -108,6 +108,9 @@ void arch_setup_new_exec(void);
 				 _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \
 				 _TIF_SYSCALL_EMU)
 
+#define _TIF_SYSCALL_EXIT_WORK	(_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \
+				 _TIF_SYSCALL_TRACEPOINT)
+
 #ifdef CONFIG_SHADOW_CALL_STACK
 #define INIT_SCS							\
 	.scs_base	= init_shadow_call_stack,			\

index 043f48ddc168..4c2bdd37d67d 100644
@@ -28,7 +28,6 @@
 #include <linux/hw_breakpoint.h>
 #include <linux/regset.h>
 #include <linux/elf.h>
-#include <linux/rseq.h>
 
 #include <asm/compat.h>
 #include <asm/cpufeature.h>

@@ -2507,12 +2506,10 @@ static inline bool report_single_step(unsigned long flags)
 	return flags & _TIF_SINGLESTEP;
 }
 
-void syscall_trace_exit(struct pt_regs *regs, unsigned long flags)
+void arm64_syscall_exit_work(struct pt_regs *regs, unsigned long flags)
 {
 	bool step;
 
-	rseq_syscall(regs);
-
 	audit_syscall_exit(regs);
 
 	if (flags & _TIF_SYSCALL_TRACEPOINT)

2.34.1

The syscall exit path in el0_svc_common() re-evaluates all _TIF_SYSCALL_WORK flags, but this mask contains flags that only matter on entry:

- _TIF_SECCOMP: seccomp filtering is entry-only
- _TIF_SYSCALL_EMU: PTRACE_SYSEMU skips the syscall on entry

Re-checking them on exit is unnecessary and may trigger redundant work.

Switch to _TIF_SYSCALL_EXIT_WORK for the exit-path re-check to evaluate only exit-relevant flags (_TIF_SYSCALL_TRACE, _TIF_SYSCALL_AUDIT, and _TIF_SYSCALL_TRACEPOINT).

No functional change intended.

---
 arch/arm64/kernel/syscall.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 0061fc63e7ba..f1f0b6f3e32b 100644
@@ -127,7 +127,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 	 */
 	if (!has_syscall_work(flags) && !IS_ENABLED(CONFIG_DEBUG_RSEQ)) {
 		flags = read_thread_flags();
-		if (has_syscall_work(flags) || flags & _TIF_SINGLESTEP)
+		if (unlikely(flags & _TIF_SYSCALL_EXIT_WORK) || flags & _TIF_SINGLESTEP)
 			arm64_syscall_exit_to_user_mode_work(regs);
 		return;
 	}

2.34.1

The fast exit path of el0_svc_common() is only taken when no syscall work was pending on entry and CONFIG_DEBUG_RSEQ is disabled:

	if (!(flags & _TIF_SYSCALL_WORK) && !IS_ENABLED(CONFIG_DEBUG_RSEQ)) {
		flags = read_thread_flags();
		if (unlikely(flags & _TIF_SYSCALL_EXIT_WORK) || flags & _TIF_SINGLESTEP)
			arm64_syscall_exit_to_user_mode_work(regs);
		return;
	}
	trace_exit:
		arm64_syscall_exit_to_user_mode_work(regs);

arm64_syscall_exit_to_user_mode_work() already re-reads the thread flags and checks the same flags itself. The only other thing it does is rseq_syscall(), which is an empty stub when CONFIG_DEBUG_RSEQ is disabled, i.e. whenever the fast path is taken. So both paths end up doing exactly the same thing, and the fast path can be dropped:

	trace_exit:
		arm64_syscall_exit_to_user_mode_work(regs);

With CONFIG_DEBUG_RSEQ enabled the fast path was never taken, so nothing changes there either.

has_syscall_work() then has a single user left, so open-code it.

No functional changes intended.

---
 arch/arm64/kernel/syscall.c | 20 +-------------------
 1 file changed, 1 insertion(+), 19 deletions(-)

index f1f0b6f3e32b..3488afd45d20 100644
@@ -54,11 +54,6 @@ static void invoke_syscall(struct pt_regs *regs, unsigned int scno,
 	syscall_set_return_value(current, regs, 0, ret);
 }
 
-static inline bool has_syscall_work(unsigned long flags)
-{
-	return unlikely(flags & _TIF_SYSCALL_WORK);
-}
-
 static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 			   const syscall_fn_t syscall_table[])
 {

@@ -95,7 +90,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 		return;
 	}
 
-	if (has_syscall_work(flags)) {
+	if (unlikely(flags & _TIF_SYSCALL_WORK)) {
 		/*
 		 * The de-facto standard way to skip a system call using ptrace
 		 * is to set the system call to -1 (NO_SYSCALL) and set x0 to a

@@ -119,19 +114,6 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 	}
 
 	invoke_syscall(regs, scno, sc_nr, syscall_table);
-
-	/*
-	 * The tracing status may have changed under our feet, so we have to
-	 * check again. However, if we were tracing entry, then we always trace
-	 * exit regardless, as the old entry assembly did.
-	 */
-	if (!has_syscall_work(flags) && !IS_ENABLED(CONFIG_DEBUG_RSEQ)) {
-		flags = read_thread_flags();
-		if (unlikely(flags & _TIF_SYSCALL_EXIT_WORK) || flags & _TIF_SINGLESTEP)
-			arm64_syscall_exit_to_user_mode_work(regs);
-		return;
-	}
-
 trace_exit:
 	arm64_syscall_exit_to_user_mode_work(regs);
 }

2.34.1

In preparation for migrating arm64 to the generic entry infrastructure, decouple the decision to run the system call from the syscall number. arm64_syscall_trace_enter() now returns a bool, like generic entry's syscall_trace_enter(), instead of overloading NO_SYSCALL (-1):

- false: do not run the system call (ptrace or seccomp rejected it)
- true: run it, after re-reading the syscall number from regs

When it returns true, el0_svc_common() re-reads the syscall number and still skips invoke_syscall() if it is NO_SYSCALL. A tracer skips a system call by setting the number to -1 and its own return value in x0; calling invoke_syscall() would overwrite that x0 with -ENOSYS from sys_ni_syscall().

The user-visible behaviour does not change:

- syscall(-1) issued by userspace still returns -ENOSYS.
- A tracer skip with a custom x0 still returns that x0.
- A tracer skip without setting x0 still returns the original x0.
- Normal system calls and seccomp rejections are unaffected.

No functional changes intended.

---
 arch/arm64/include/asm/syscall.h | 2 +-
 arch/arm64/kernel/ptrace.c       | 8 ++++----
 arch/arm64/kernel/syscall.c      | 6 +++++-
 3 files changed, 10 insertions(+), 6 deletions(-)

index 7cc872fde019..f19d4a8824ba 100644
@@ -121,7 +121,7 @@ static inline int syscall_get_arch(struct task_struct *task)
 	return AUDIT_ARCH_AARCH64;
 }
 
-int arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags);
+bool arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags);
 void arm64_syscall_exit_work(struct pt_regs *regs, unsigned long flags);
 
 static __always_inline void arm64_syscall_exit_to_user_mode_work(struct pt_regs *regs)

index 4c2bdd37d67d..d3c52c9a1105 100644
@@ -2470,14 +2470,14 @@ static inline void syscall_enter_audit(struct pt_regs *regs)
 static inline void syscall_enter_audit(struct pt_regs *regs) { }
 #endif
 
-int arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags)
+bool arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags)
 {
 	int ret;
 
 	if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) {
 		ret = report_syscall_entry(regs);
 		if (ret || (flags & _TIF_SYSCALL_EMU))
-			return NO_SYSCALL;
+			return false;
 
 		/* ptrace might have changed thread flags */
 		flags = read_thread_flags();

@@ -2486,7 +2486,7 @@ int arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags)
 	/* Do the secure computing after ptrace; failures should be fast. */
 	if (unlikely(flags & _TIF_SECCOMP)) {
 		if (!__seccomp_permit_syscall())
-			return NO_SYSCALL;
+			return false;
 	}
 
 	if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))

@@ -2495,7 +2495,7 @@ int arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags)
 	if (unlikely(audit_context()))
 		syscall_enter_audit(regs);
 
-	return syscall_get_nr(current, regs);
+	return true;
 }
 
 static inline bool report_single_step(unsigned long flags)

index 3488afd45d20..72c6e8b7ab21 100644
@@ -108,7 +108,11 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 		 */
 		if (scno == NO_SYSCALL)
 			syscall_set_return_value(current, regs, -ENOSYS, 0);
-		scno = arm64_syscall_trace_enter(regs, read_thread_flags());
+		if (!arm64_syscall_trace_enter(regs, read_thread_flags()))
+			goto trace_exit;
+
+		/* Reread the syscall number as it might have been modified */
+		scno = syscall_get_nr(current, regs);
 		if (scno == NO_SYSCALL)
 			goto trace_exit;
 	}

2.34.1

Implement the generic entry framework for arm64 to handle system call entry and exit. This follows the migration of x86, RISC-V, and LoongArch, consolidating architecture-specific syscall tracing and auditing into the common kernel entry infrastructure, making it easier to enable advanced features on arm64, such as "Syscall User Dispatch" and "rseq time slice extension" optimization.

[Background]
Arm64 has already adopted generic IRQ entry. Completing the conversion to the generic syscall entry framework reduces architectural divergence, simplifies maintenance, and allows arm64 to automatically benefit from improvements in the common entry code.

[Changes]

1. Kconfig and Infrastructure:
- Select GENERIC_ENTRY and remove GENERIC_IRQ_ENTRY (now implied).

- Migrate struct thread_info to use the syscall_work field instead of TIF flags for syscall-related tasks.

2. Thread Info and Flags:
- Remove definitions for TIF_SYSCALL_TRACE, TIF_SYSCALL_AUDIT, TIF_SYSCALL_TRACEPOINT, TIF_SECCOMP, and TIF_SYSCALL_EMU.

- Replace _TIF_SYSCALL_WORK and _TIF_SYSCALL_EXIT_WORK with the generic SYSCALL_WORK bitmask.

- Map single-step state to SYSCALL_EXIT_TRAP in debug-monitors.c.

3. Architecture-Specific Hooks (asm/entry-common.h): Implement following arch function by porting the existing arm64 logic to the generic interface:
- arch_ptrace_report_syscall_permit_entry()
- arch_ptrace_report_syscall_exit()

4. Cleanup and Refactoring: Remove redundant arm64-specific syscall tracing functions from ptrace.c, including following key functions and related audit/step helpers:
- arm64_syscall_exit_to_user_mode_work()
- arm64_syscall_trace_enter()
- arm64_syscall_exit_work().

- Update el0_svc_common() in syscall.c to use the generic syscall_work checks and entry/exit call sites.

[Why this matters]
- Unified Interface: Aligns arm64 with the modern kernel entry standard.

- Improved Maintainability: Bug fixes in kernel/entry/common.c now apply to arm64 automatically.

- Feature Readiness: Simplifies the implementation of future cross-architecture syscall features.

[Compatibility]
This conversion maintains full ABI compatibility with existing userspace. The ptrace register-saving behavior, seccomp filtering, and syscall tracing semantics remain identical to the previous implementation.

The set of traceable functions on the syscall path does change, though this is not ABI. Removed: syscall_trace_enter(), syscall_trace_exit() and el0_svc_common.constprop.0. Added: trace_syscall_enter(), trace_syscall_exit() and syscall_enter_audit(). The new names are the same ones x86, RISC-V, LoongArch and s390 already expose.

Note that Syscall User Dispatch and the rseq time slice extension are still not enabled on arm64 by this series: they additionally require ARCH_SUPPORTS_SYSCALL_USER_DISPATCH and HAVE_GENERIC_TIF_BITS respectively, neither of which arm64 selects yet.

Additionally, since arm64 now does not select HAVE_GENERIC_TIF_BITS, there is no functional change regarding rseq management, as those generic entry pathways remain a no-op for this architecture.

CONFIG_DEBUG_RSEQ depends on !GENERIC_ENTRY, so it can no longer be enabled on arm64. Generic entry does the same check in rseq_debug_syscall_return(), controlled by the rseq_debug= boot option and debugfs, and also calls it before any ptrace stop at syscall exit.

---
 arch/arm64/Kconfig                    |   2 +-
 arch/arm64/include/asm/entry-common.h |  77 ++++++++++++++
 arch/arm64/include/asm/syscall.h      |  14 ---
 arch/arm64/include/asm/thread_info.h  |  19 +---
 arch/arm64/kernel/debug-monitors.c    |   8 ++
 arch/arm64/kernel/entry-common.c      |   2 +-
 arch/arm64/kernel/ptrace.c            | 138 --------------------------
 arch/arm64/kernel/signal.c            |   2 +-
 arch/arm64/kernel/syscall.c           |   9 +-
 9 files changed, 95 insertions(+), 176 deletions(-)

index 1075bc661833..860d8ae035de 100644
@@ -129,9 +129,9 @@ config ARM64
 	select GENERIC_CPU_DEVICES
 	select GENERIC_CPU_VULNERABILITIES
 	select GENERIC_EARLY_IOREMAP
+	select GENERIC_ENTRY
 	select GENERIC_IDLE_POLL_SETUP
 	select GENERIC_IOREMAP
-	select GENERIC_IRQ_ENTRY
 	select GENERIC_IRQ_IPI
 	select GENERIC_IRQ_KEXEC_CLEAR_VM_FORWARD
 	select GENERIC_IRQ_PROBE

index f19d4a8824ba..e3e716575846 100644
@@ -120,18 +120,4 @@ static inline int syscall_get_arch(struct task_struct *task)
 
 	return AUDIT_ARCH_AARCH64;
 }
-
-bool arm64_syscall_trace_enter(struct pt_regs *regs, unsigned long flags);
-void arm64_syscall_exit_work(struct pt_regs *regs, unsigned long flags);
-
-static __always_inline void arm64_syscall_exit_to_user_mode_work(struct pt_regs *regs)
-{
-	unsigned long flags = read_thread_flags();
-
-	rseq_syscall(regs);
-
-	if (unlikely(flags & _TIF_SYSCALL_EXIT_WORK) || flags & _TIF_SINGLESTEP)
-		arm64_syscall_exit_work(regs, flags);
-}
-
 #endif	/* __ASM_SYSCALL_H */

index a85c72b65b82..855c8fd13f81 100644
@@ -42,6 +42,7 @@ struct thread_info {
 #endif
 	u32			cpu;
 	u16			pcpu_gprs;
+	unsigned long		syscall_work;   /* SYSCALL_WORK_ flags */
 };
 
 #define thread_saved_pc(tsk)	\

@@ -64,11 +65,6 @@ void arch_setup_new_exec(void);
 #define TIF_UPROBE		5	/* uprobe breakpoint or singlestep */
 #define TIF_MTE_ASYNC_FAULT	6	/* MTE Asynchronous Tag Check Fault */
 #define TIF_NOTIFY_SIGNAL	7	/* signal notifications exist */
-#define TIF_SYSCALL_TRACE	8	/* syscall trace active */
-#define TIF_SYSCALL_AUDIT	9	/* syscall auditing */
-#define TIF_SYSCALL_TRACEPOINT	10	/* syscall tracepoint for ftrace */
-#define TIF_SECCOMP		11	/* syscall secure computing */
-#define TIF_SYSCALL_EMU		12	/* syscall emulation active */
 #define TIF_PATCH_PENDING	13	/* pending live patching update */
 #define TIF_MEMDIE		18	/* is terminating due to OOM killer */
 #define TIF_FREEZE		19

@@ -90,27 +86,14 @@ void arch_setup_new_exec(void);
 #define _TIF_NEED_RESCHED_LAZY	(1 << TIF_NEED_RESCHED_LAZY)
 #define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_FOREIGN_FPSTATE	(1 << TIF_FOREIGN_FPSTATE)
-#define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
-#define _TIF_SYSCALL_AUDIT	(1 << TIF_SYSCALL_AUDIT)
-#define _TIF_SYSCALL_TRACEPOINT	(1 << TIF_SYSCALL_TRACEPOINT)
-#define _TIF_SECCOMP		(1 << TIF_SECCOMP)
-#define _TIF_SYSCALL_EMU	(1 << TIF_SYSCALL_EMU)
 #define _TIF_PATCH_PENDING	(1 << TIF_PATCH_PENDING)
 #define _TIF_UPROBE		(1 << TIF_UPROBE)
-#define _TIF_SINGLESTEP		(1 << TIF_SINGLESTEP)
 #define _TIF_32BIT		(1 << TIF_32BIT)
 #define _TIF_SVE		(1 << TIF_SVE)
 #define _TIF_MTE_ASYNC_FAULT	(1 << TIF_MTE_ASYNC_FAULT)
 #define _TIF_NOTIFY_SIGNAL	(1 << TIF_NOTIFY_SIGNAL)
 #define _TIF_TSC_SIGSEGV	(1 << TIF_TSC_SIGSEGV)
 
-#define _TIF_SYSCALL_WORK	(_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \
-				 _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \
-				 _TIF_SYSCALL_EMU)
-
-#define _TIF_SYSCALL_EXIT_WORK	(_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \
-				 _TIF_SYSCALL_TRACEPOINT)
-
 #ifdef CONFIG_SHADOW_CALL_STACK
 #define INIT_SCS							\
 	.scs_base	= init_shadow_call_stack,			\

index e271fbac5f82..5da530034e2e 100644
@@ -382,11 +382,19 @@ void user_enable_single_step(struct task_struct *task)
 
 	if (!test_and_set_ti_thread_flag(ti, TIF_SINGLESTEP))
 		set_regs_spsr_ss(task_pt_regs(task));
+
+	/*
+	 * Ensure that the generic entry code triggers a trap once stepping
+	 * out of a system call prior to executing any user instruction,
+	 * as the generic entry code does not natively check for TIF_SINGLESTEP.
+	 */
+	set_task_syscall_work(task, SYSCALL_EXIT_TRAP);
 }
 NOKPROBE_SYMBOL(user_enable_single_step);
 
 void user_disable_single_step(struct task_struct *task)
 {
 	clear_ti_thread_flag(task_thread_info(task), TIF_SINGLESTEP);
+	clear_task_syscall_work(task, SYSCALL_EXIT_TRAP);
 }
 NOKPROBE_SYMBOL(user_disable_single_step);

index 11d3f888fcd8..121f4d6d99d1 100644
@@ -6,7 +6,7 @@
  */
 
 #include <linux/context_tracking.h>
-#include <linux/irq-entry-common.h>
+#include <linux/entry-common.h>
 #include <linux/kasan.h>
 #include <linux/linkage.h>
 #include <linux/livepatch.h>

index 6a06a4e352b1..9c51263fb01b 100644
@@ -8,8 +8,8 @@
 
 #include <linux/cache.h>
 #include <linux/compat.h>
+#include <linux/entry-common.h>
 #include <linux/errno.h>
-#include <linux/irq-entry-common.h>
 #include <linux/kernel.h>
 #include <linux/signal.h>
 #include <linux/freezer.h>

index 72c6e8b7ab21..4a13108defb0 100644
@@ -2,6 +2,7 @@
 
 #include <linux/compiler.h>
 #include <linux/context_tracking.h>
+#include <linux/entry-common.h>
 #include <linux/errno.h>
 #include <linux/nospec.h>
 #include <linux/ptrace.h>

@@ -58,6 +59,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 			   const syscall_fn_t syscall_table[])
 {
 	unsigned long flags = read_thread_flags();
+	unsigned long work;
 
 	regs->orig_x0 = regs->regs[0];
 	regs->syscallno = scno;

@@ -90,7 +92,8 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 		return;
 	}
 
-	if (unlikely(flags & _TIF_SYSCALL_WORK)) {
+	work = READ_ONCE(current_thread_info()->syscall_work);
+	if (unlikely(work & SYSCALL_WORK_ENTER)) {
 		/*
 		 * The de-facto standard way to skip a system call using ptrace
 		 * is to set the system call to -1 (NO_SYSCALL) and set x0 to a

@@ -108,7 +111,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 		 */
 		if (scno == NO_SYSCALL)
 			syscall_set_return_value(current, regs, -ENOSYS, 0);
-		if (!arm64_syscall_trace_enter(regs, read_thread_flags()))
+		if (!syscall_trace_enter(regs, work, scno))
 			goto trace_exit;
 
 		/* Reread the syscall number as it might have been modified */

@@ -119,7 +122,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
 
 	invoke_syscall(regs, scno, sc_nr, syscall_table);
 trace_exit:
-	arm64_syscall_exit_to_user_mode_work(regs);
+	syscall_exit_to_user_mode_work(regs);
 }
 
 void do_el0_svc(struct pt_regs *regs)

2.34.1

After converting arm64 to Generic Entry framework, the compiler no longer inlines el0_svc_common() into its caller do_el0_svc(). This introduces a small but measurable overhead in the critical system call path.

Manually forcing el0_svc_common() to be inlined restores the performance. Benchmarking with perf bench syscall basic on a Kunpeng 920 platform (based on v6.19-rc1) shows a ~1% performance uplift.

Inlining this function reduces function prologue/epilogue overhead and allows for better compiler optimization in the hot system call dispatch path.

| Metric     | W/O this patch | With this patch | Change    |
| ---------- | -------------- | --------------- | --------- |
| Total time | 2.195 [sec]    | 2.171 [sec]     |  ↓1.1%   |
| usecs/op   | 0.219575       | 0.217192        |  ↓1.1%   |
| ops/sec    | 4,554,260      | 4,604,225       |  ↑1.1%    |

Two notes on the inlining:

- Syscall-path stack usage grows slightly. do_el0_svc() now has a single 112-byte frame and 13 calls, where previously a 16-byte frame called a 48-byte el0_svc_common(); that is +48 bytes of stack on the syscall path. lkdtm's KSTACK_ERASE confirms the 48-byte delta at runtime.

- do_el0_svc() is not noinstr and stays in .text, so the inlined code is instrumentable, as el0_svc_common() was before. This is fine:
el0_svc() only calls do_el0_svc() after arm64_syscall_enter_from_user_mode() has set up the context tracking, lockdep and RCU state, and do_el0_svc() is in a different translation unit, so it is not inlined into the noinstr el0_svc().

---
 arch/arm64/kernel/syscall.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 4a13108defb0..2535cae9413d 100644
@@ -55,8 +55,8 @@ static void invoke_syscall(struct pt_regs *regs, unsigned int scno,
 	syscall_set_return_value(current, regs, 0, ret);
 }
 
-static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
-			   const syscall_fn_t syscall_table[])
+static __always_inline void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,
+					   const syscall_fn_t syscall_table[])
 {
 	unsigned long flags = read_thread_flags();
 	unsigned long work;

2.34.1

Add tests for the arm64 syscall-exit and single-step behaviour fixed by the following commits. Each test fails on a kernel without the fix it covers and passes with it.

pseudo-step fix: commit ac2081cdc4d99 ("arm64: ptrace: Consistently
use pseudo-singlestep exceptions")
SPSR.SS override fix: commit 3a5a4366cecc ("arm64: ptrace: Override
SPSR.SS when single-stepping is enabled")
SYSEMU fix: "arm64: ptrace: Fix spurious pseudo-single-step
for PTRACE_SYSEMU_SINGLESTEP" (this series)
rseq fix: "arm64: ptrace: Check rseq critical section
before reporting syscall exit" (this series)

single_step_signal_handler
Single-step into a SIGUSR1 handler while SIGTRAP is blocked and has its own handler. The tracee must stop at the handler entry, SIGTRAP must still be blocked in the handler, and its handler must not be reset to SIG_DFL.
Covers the pseudo-step fix, issue reported by Keno Fischer.

single_step_spsr_ss
Clear SPSR.SS with PTRACE_SETREGSET, as GDB does when reverse-stepping, then PTRACE_SINGLESTEP. The PC must move by one instruction. Covers the SPSR.SS override fix, issue reported by Luis Machado.

single_step_syscall_stop
Single-step across a system call that takes a PTRACE_EVENT_SECCOMP stop. The first stop after the seccomp stop must be at the SVC return address, both when the tracer continued to the seccomp stop and when it single-stepped into it and rewrote the registers there. Covers the pseudo-step fix; the second case is the PTRACE_SETREGSET variant Mark Rutland pointed out, which only shows up once the SPSR.SS override fix is applied.

sysemu_singlestep
PTRACE_SYSEMU_SINGLESTEP over a system call: no syscall-exit-stop (ptrace(2)), no pseudo-step stop after the syscall-enter-stop, and no pseudo-step after resuming a PTRACE_EVENT_SECCOMP stop with PTRACE_SYSEMU_SINGLESTEP.
Covers the SYSEMU fix.

rseq_syscall_pc
With rseq debugging enabled, a tracer moves the PC at the syscall-exit-stop. A system call issued inside an rseq critical section must still kill the task with SIGSEGV after the PC is moved out, and one issued outside must not after the PC is moved in. Skipped if rseq debugging is not enabled (CONFIG_DEBUG_RSEQ before generic entry, the rseq_debug= boot option after). Covers the rseq fix.

Results in QEMU (-M virt -cpu cortex-a57), P = pass, F = fail, S = skip, one letter per test point:

  kernel                              sig  spsr sys  emu  rseq
  v5.8-rc1+, before the pseudo-step   PFF  F    FP   PPP  S
  + pseudo-step fix                   PPP  F    PP   PFF  S
  + SPSR.SS override fix only         PFF  P    FF   PPP  S
  + both                              PPP  P    PP   PFF  S
  v7.4-rc1                            PPP  P    PP   PFF  S
  + SYSEMU fix, CONFIG_DEBUG_RSEQ=y   PPP  P    PP   PPP  FF
  + rseq fix,   CONFIG_DEBUG_RSEQ=y   PPP  P    PP   PPP  PP
  this series                         PPP  P    PP   PPP  S
  this series, rseq_debug=1           PPP  P    PP   PPP  PP

"SPSR.SS override fix only" is the SPSR.SS override fix cherry-picked onto the parent of the pseudo-step fix. That combination never existed in the tree, but it is the state in which the PTRACE_SETREGSET case of single_step_syscall_stop fails.

---
 tools/testing/selftests/arm64/Makefile        |   2 +-
 tools/testing/selftests/arm64/ptrace/Makefile |  12 ++
 .../selftests/arm64/ptrace/ptrace_common.h    | 101 ++++++++++
 .../selftests/arm64/ptrace/rseq_syscall_pc.c  | 130 +++++++++++++
 .../selftests/arm64/ptrace/rseq_syscall_pc.h  |  22 +++
 .../arm64/ptrace/rseq_syscall_pc_asm.S        |  45 +++++
 .../arm64/ptrace/single_step_signal_handler.c | 121 ++++++++++++
 .../arm64/ptrace/single_step_spsr_ss.c        |  79 ++++++++
 .../arm64/ptrace/single_step_syscall_stop.c   | 138 ++++++++++++++
 .../arm64/ptrace/sysemu_singlestep.c          | 172 ++++++++++++++++++
 10 files changed, 821 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/arm64/ptrace/Makefile
 create mode 100644 tools/testing/selftests/arm64/ptrace/ptrace_common.h
 create mode 100644 tools/testing/selftests/arm64/ptrace/rseq_syscall_pc.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/rseq_syscall_pc.h
 create mode 100644 tools/testing/selftests/arm64/ptrace/rseq_syscall_pc_asm.S
 create mode 100644 tools/testing/selftests/arm64/ptrace/single_step_signal_handler.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/single_step_spsr_ss.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/single_step_syscall_stop.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/sysemu_singlestep.c

index e456f3b62fa1..3edd476f8fc0 100644
@@ -4,7 +4,7 @@
 ARCH ?= $(shell uname -m 2>/dev/null || echo not)
 
 ifneq (,$(filter $(ARCH),aarch64 arm64))
-ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs
+ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs ptrace
 else
 ARM64_SUBTARGETS :=
 endif

new file mode 100644
index 000000000000..74c250120a2f
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0
+CFLAGS += -Wall -O2 -g $(KHDR_INCLUDES) -I$(top_srcdir)/tools/include
+
+TEST_GEN_PROGS := single_step_signal_handler \
+		  single_step_spsr_ss \
+		  single_step_syscall_stop \
+		  sysemu_singlestep \
+		  rseq_syscall_pc
+
+include ../../lib.mk
+
+$(OUTPUT)/rseq_syscall_pc: rseq_syscall_pc.c rseq_syscall_pc_asm.S

new file mode 100644
index 000000000000..975fb883810f
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef RSEQ_SYSCALL_PC_H
+#define RSEQ_SYSCALL_PC_H
+
+/* BRK #0x45E0, the arm64 RSEQ_SIG */
+#define RSEQ_SIG_INSN		0xd428bc00
+
+#ifndef __ASSEMBLER__
+/*
+ * Both functions store @cs into @slot (rseq->rseq_cs) and then issue
+ * getppid(). call_in_cs() does that inside the critical section
+ * [cs_start, cs_end), call_out_of_cs() outside it. Both return 0, or 42
+ * if the critical section was aborted.
+ */
+long call_in_cs(void *slot, void *cs);
+long call_out_of_cs(void *slot, void *cs);
+
+extern char cs_start[], cs_after_svc[], cs_end[], cs_abort[];
+extern char out_after_svc[];
+#endif
+
+#endif /* RSEQ_SYSCALL_PC_H */

2.34.1