Skip to contentSkip to conversation

arm64/ptrace: allow to get all registers on syscall traps

PATCH Superseded Andrei Vagin 5 participants 15 messages Feb 1, 2021 – Feb 25, 2021

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover arm64/ptrace: allow to get all registers on syscall traps

Right now, ip/r12 for AArch32 and x7 for AArch64 is used to indicate whether or not the stop has been signalled from syscall entry or syscall exit. This means that:

- Any writes by the tracer to this register during the stop are ignored/discarded.

- The actual value of the register is not available during the stop, so the tracer cannot save it and restore it later.

For applications like the user-mode Linux or gVisor, it is critical to have access to the full set of registers in any moment. For example, they need to change values of all registers to emulate rt_sigreturn or execve and they need to have the full set of registers to build a signal frame.

This series introduces the PTRACE_O_ARM64_RAW_REGS option. If it is set, PTRACE_GETREGSET returns values of all registers, and PTRACE_SETREGSET allows to change any of them.

Cc Catalin Marinas <[email protected]>
Cc Dave Martin <[email protected]>
Cc Keno Fischer <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Will Deacon <[email protected]>

Andrei Vagin (3):
arm64/ptrace: don't clobber task registers on syscall entry/exit traps arm64/ptrace: introduce PTRACE_O_ARM64_RAW_REGS selftest/arm64/ptrace: add tests for PTRACE_O_ARM64_RAW_REGS

v2: use the ptrace option instead of adding a new regset.

 arch/arm64/include/asm/ptrace.h               |   5 +
 arch/arm64/kernel/ptrace.c                    | 130 +++++++++++-----
 include/uapi/linux/elf.h                      |   1 +
 tools/testing/selftests/arm64/Makefile        |   2 +-
 tools/testing/selftests/arm64/ptrace/Makefile |   6 +
 .../arm64/ptrace/ptrace_syscall_regs_test.c   | 142 ++++++++++++++++++
 6 files changed, 246 insertions(+), 40 deletions(-)
 create mode 100644 tools/testing/selftests/arm64/ptrace/Makefile
 create mode 100644 tools/testing/selftests/arm64/ptrace/ptrace_syscall_regs_test.c
Signature · 1 line
2.29.2
#1 lore raw

PATCH v2 1/3 arm64/ptrace: don't clobber task registers on syscall entry/exit traps

ip/r12 for AArch32 and x7 for AArch64 is used to indicate whether or not the stop has been signalled from syscall entry or syscall exit. This means that:

- Any writes by the tracer to this register during the stop are ignored/discarded.

- The actual value of the register is not available during the stop, so the tracer cannot save it and restore it later.

Right now, these registers are clobbered in tracehook_report_syscall. This change moves the logic to gpr_get and compat_gpr_get where registers are copied into a user-space buffer.

This will allow to change these registers and to introduce a new ptrace option to get the full set of registers.

---
 arch/arm64/include/asm/ptrace.h |   5 ++
 arch/arm64/kernel/ptrace.c      | 104 ++++++++++++++++++++------------
 2 files changed, 69 insertions(+), 40 deletions(-)
arch/arm64/include/asm/ptrace.h +5 -0
index e58bca832dff..0a9552b4f61e 100644
@@ -170,6 +170,11 @@ static inline unsigned long pstate_to_compat_psr(const unsigned long pstate)
 	return psr;
 }
 
+enum ptrace_syscall_dir {
+	PTRACE_SYSCALL_ENTER = 0,
+	PTRACE_SYSCALL_EXIT,
+};
+
 /*
  * This struct defines the way the registers are stored on the stack during an
  * exception. Note that sizeof(struct pt_regs) has to be a multiple of 16 (for
arch/arm64/kernel/ptrace.c +64 -40

148 lines not on this page · raw message

Signature · 1 line
2.29.2
maintainer
#2 lore raw

Re: [PATCH 1/3] arm64/ptrace: don't clobber task registers on syscall entry/exit traps

Quoted 55 lines · Andrei Vagin

ip/r12 for AArch32 and x7 for AArch64 is used to indicate whether or not the stop has been signalled from syscall entry or syscall exit. This means that:

- Any writes by the tracer to this register during the stop are ignored/discarded.

- The actual value of the register is not available during the stop, so the tracer cannot save it and restore it later.

Right now, these registers are clobbered in tracehook_report_syscall. This change moves the logic to gpr_get and compat_gpr_get where registers are copied into a user-space buffer.

This will allow to change these registers and to introduce a new ptrace option to get the full set of registers.

---
 arch/arm64/include/asm/ptrace.h |   5 ++
 arch/arm64/kernel/ptrace.c      | 104 ++++++++++++++++++++------------
 2 files changed, 69 insertions(+), 40 deletions(-)
arch/arm64/include/asm/ptrace.h +5 -0
index e58bca832dff..0a9552b4f61e 100644
@@ -170,6 +170,11 @@ static inline unsigned long pstate_to_compat_psr(const unsigned long pstate)
 	return psr;
 }
 
+enum ptrace_syscall_dir {
+	PTRACE_SYSCALL_ENTER = 0,
+	PTRACE_SYSCALL_EXIT,
+};
+
 /*
  * This struct defines the way the registers are stored on the stack during an
  * exception. Note that sizeof(struct pt_regs) has to be a multiple of 16 (for
arch/arm64/kernel/ptrace.c +1 -0
index 8ac487c84e37..39da03104528 100644
@@ -40,6 +40,7 @@
 #include <asm/syscall.h>
 #include <asm/traps.h>
 #include <asm/system_misc.h>
+#include <asm/ptrace.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/syscalls.h>
@@ -561,7 +562,31 @@ static int gpr_get(struct task_struct *target,
 		   struct membuf to)
 {
 	struct user_pt_regs *uregs = &task_pt_regs(target)->user_regs;

arch/arm64/kernel/ptrace.c static int gpr_get(struct task_struct *target, · line 573

Quoted 10 lines
arch/arm64/kernel/ptrace.c +9 -1
-	return membuf_write(&to, uregs, sizeof(*uregs));
+	unsigned long saved_reg;
+	int ret;
+
+	/*
+	 * We have some ABI weirdness here in the way that we handle syscall
+	 * exit stops because we indicate whether or not the stop has been
+	 * signalled from syscall entry or syscall exit by clobbering the general
+	 * purpose register x7.
+	 */

When you move a comment, please don't truncate it!

arch/arm64/kernel/ptrace.c static int gpr_get(struct task_struct *target, · line 583

Quoted 10 lines
arch/arm64/kernel/ptrace.c +10 -0
+	saved_reg = uregs->regs[7];
+
+	switch (target->ptrace_message) {
+	case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+		uregs->regs[7] = PTRACE_SYSCALL_ENTER;
+		break;
+	case PTRACE_EVENTMSG_SYSCALL_EXIT:
+		uregs->regs[7] = PTRACE_SYSCALL_EXIT;
+		break;
+	}

I'm wary of checking target->ptrace_message here, as I seem to recall the regset code also being used for coredumps. What guarantees we don't break things there?

Quoted 109 lines
arch/arm64/kernel/ptrace.c +42 -32
+
+	ret =  membuf_write(&to, uregs, sizeof(*uregs));
+
+	uregs->regs[7] = saved_reg;
+
+	return ret;
 }
 
 static int gpr_set(struct task_struct *target, const struct user_regset *regset,
@@ -575,6 +600,17 @@ static int gpr_set(struct task_struct *target, const struct user_regset *regset,
 	if (ret)
 		return ret;
 
+	/*
+	 * Historically, x7 can't be changed if the stop has been signalled
+	 * from syscall-enter of syscall-exit.
+	 */
+	switch (target->ptrace_message) {
+	case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+	case PTRACE_EVENTMSG_SYSCALL_EXIT:
+		newregs.regs[7] = task_pt_regs(target)->regs[7];
+		break;
+	}
+
 	if (!valid_user_regs(&newregs, target))
 		return -EINVAL;
 
@@ -1206,6 +1242,20 @@ static inline compat_ulong_t compat_get_user_reg(struct task_struct *task, int i
 	struct pt_regs *regs = task_pt_regs(task);
 
 	switch (idx) {
+	case 12:
+		/*
+		 * We have some ABI weirdness here in the way that we handle
+		 * syscall exit stops because we indicate whether or not the
+		 * stop has been signalled from syscall entry or syscall exit
+		 * by clobbering the general purpose register r12.
+		 */
+		switch (task->ptrace_message) {
+		case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+			return PTRACE_SYSCALL_ENTER;
+		case PTRACE_EVENTMSG_SYSCALL_EXIT:
+			return PTRACE_SYSCALL_EXIT;
+		}
+		return regs->regs[idx];
 	case 15:
 		return regs->pc;
 	case 16:
@@ -1282,6 +1332,17 @@ static int compat_gpr_set(struct task_struct *target,
 
 	}
 
+	/*
+	 * Historically, x12 can't be changed if the stop has been signalled
+	 * from syscall-enter of syscall-exit.
+	 */
+	switch (target->ptrace_message) {
+	case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+	case PTRACE_EVENTMSG_SYSCALL_EXIT:
+		newregs.regs[12] = task_pt_regs(target)->regs[12];
+		break;
+	}
+
 	if (valid_user_regs(&newregs.user_regs, target))
 		*task_pt_regs(target) = newregs;
 	else
@@ -1740,53 +1801,16 @@ long arch_ptrace(struct task_struct *child, long request,
 	return ptrace_request(child, request, addr, data);
 }
 
-enum ptrace_syscall_dir {
-	PTRACE_SYSCALL_ENTER = 0,
-	PTRACE_SYSCALL_EXIT,
-};
-
 static void tracehook_report_syscall(struct pt_regs *regs,
 				     enum ptrace_syscall_dir dir)
 {
-	int regno;
-	unsigned long saved_reg;
-
-	/*
-	 * We have some ABI weirdness here in the way that we handle syscall
-	 * exit stops because we indicate whether or not the stop has been
-	 * signalled from syscall entry or syscall exit by clobbering a general
-	 * purpose register (ip/r12 for AArch32, x7 for AArch64) in the tracee
-	 * and restoring its old value after the stop. This means that:
-	 *
-	 * - Any writes by the tracer to this register during the stop are
-	 *   ignored/discarded.
-	 *
-	 * - The actual value of the register is not available during the stop,
-	 *   so the tracer cannot save it and restore it later.
-	 *
-	 * - Syscall stops behave differently to seccomp and pseudo-step traps
-	 *   (the latter do not nobble any registers).
-	 */
-	regno = (is_compat_task() ? 12 : 7);
-	saved_reg = regs->regs[regno];
-	regs->regs[regno] = dir;
-
 	if (dir == PTRACE_SYSCALL_ENTER) {
 		if (tracehook_report_syscall_entry(regs))
 			forget_syscall(regs);
-		regs->regs[regno] = saved_reg;
-	} else if (!test_thread_flag(TIF_SINGLESTEP)) {
-		tracehook_report_syscall_exit(regs, 0);
-		regs->regs[regno] = saved_reg;
 	} else {

arch/arm64/kernel/ptrace.c long arch_ptrace(struct task_struct *child, long request, · line 1813

Quoted 10 lines
arch/arm64/kernel/ptrace.c +2 -7
-		regs->regs[regno] = saved_reg;
+		int singlestep = test_thread_flag(TIF_SINGLESTEP);
 
-		/*
-		 * Signal a pseudo-step exception since we are stepping but
-		 * tracer modifications to the registers may have rewound the
-		 * state machine.
-		 */
-		tracehook_report_syscall_exit(regs, 1);
+		tracehook_report_syscall_exit(regs, singlestep);

Again, please preserve the comment in some form (maybe "... if we are stepping since tracer ...").

That said, doesn't your change above break the pseudo-step trap? Currently, we report the real x7 for those.

Will

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#3 lore raw

Re: [PATCH 1/3] arm64/ptrace: don't clobber task registers on syscall entry/exit traps

Quoted 69 lines · Will Deacon
Quoted 55 lines · Andrei Vagin

ip/r12 for AArch32 and x7 for AArch64 is used to indicate whether or not the stop has been signalled from syscall entry or syscall exit. This means that:

- Any writes by the tracer to this register during the stop are ignored/discarded.

- The actual value of the register is not available during the stop, so the tracer cannot save it and restore it later.

Right now, these registers are clobbered in tracehook_report_syscall. This change moves the logic to gpr_get and compat_gpr_get where registers are copied into a user-space buffer.

This will allow to change these registers and to introduce a new ptrace option to get the full set of registers.

---
 arch/arm64/include/asm/ptrace.h |   5 ++
 arch/arm64/kernel/ptrace.c      | 104 ++++++++++++++++++++------------
 2 files changed, 69 insertions(+), 40 deletions(-)
arch/arm64/include/asm/ptrace.h +5 -0
index e58bca832dff..0a9552b4f61e 100644
@@ -170,6 +170,11 @@ static inline unsigned long pstate_to_compat_psr(const unsigned long pstate)
 	return psr;
 }
 
+enum ptrace_syscall_dir {
+	PTRACE_SYSCALL_ENTER = 0,
+	PTRACE_SYSCALL_EXIT,
+};
+
 /*
  * This struct defines the way the registers are stored on the stack during an
  * exception. Note that sizeof(struct pt_regs) has to be a multiple of 16 (for
arch/arm64/kernel/ptrace.c +1 -0
index 8ac487c84e37..39da03104528 100644
@@ -40,6 +40,7 @@
 #include <asm/syscall.h>
 #include <asm/traps.h>
 #include <asm/system_misc.h>
+#include <asm/ptrace.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/syscalls.h>
@@ -561,7 +562,31 @@ static int gpr_get(struct task_struct *target,
 		   struct membuf to)
 {
 	struct user_pt_regs *uregs = &task_pt_regs(target)->user_regs;

arch/arm64/kernel/ptrace.c static int gpr_get(struct task_struct *target, · line 573

Quoted 10 lines
arch/arm64/kernel/ptrace.c +9 -1
-	return membuf_write(&to, uregs, sizeof(*uregs));
+	unsigned long saved_reg;
+	int ret;
+
+	/*
+	 * We have some ABI weirdness here in the way that we handle syscall
+	 * exit stops because we indicate whether or not the stop has been
+	 * signalled from syscall entry or syscall exit by clobbering the general
+	 * purpose register x7.
+	 */

When you move a comment, please don't truncate it!

Quoted 14 lines

arch/arm64/kernel/ptrace.c static int gpr_get(struct task_struct *target, · line 583

Quoted 10 lines
arch/arm64/kernel/ptrace.c +10 -0
+	saved_reg = uregs->regs[7];
+
+	switch (target->ptrace_message) {
+	case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+		uregs->regs[7] = PTRACE_SYSCALL_ENTER;
+		break;
+	case PTRACE_EVENTMSG_SYSCALL_EXIT:
+		uregs->regs[7] = PTRACE_SYSCALL_EXIT;
+		break;
+	}

I'm wary of checking target->ptrace_message here, as I seem to recall the regset code also being used for coredumps. What guarantees we don't break things there?

For a coredump, is there any way to know whether a given thread was inside a traced syscall when the coredump was generated? If so, x7 in the dump may already unreliable and we only need to make best efforts to get it "right".

Since triggering of the coredump and death of other threads all require dequeueing of some signal, I think all threads must always outside the syscall-enter...syscall-exit path before any of the coredump runs anyway, in which case the above should never matter... Though somone else ought to eyeball the coredump code before we agree on that.

ptrace_message doesn't seem absolutely the wrong thing to check, but we'd need to be sure that it can't be stale (say, left over from some previous trap).

Out of interest, where did this arm64 ptrace feature come from? Was it just pasted from 32-bit and thinly adapted? It looks like an arch-specific attempt to do what PTRACE_O_TRACESYSGOOD does, in which case it may have been obsolete even before it was upstreamed. I wonder whether anyone is actually relying on it at all...

Doesn't mean we can definitely fix it safely, but it's annoying.

[...]

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#4 lore raw

Re: [PATCH 1/3] arm64/ptrace: don't clobber task registers on syscall entry/exit traps

In reply to Will Deacon
Quoted 60 lines · Will Deacon
Quoted 59 lines · Andrei Vagin

ip/r12 for AArch32 and x7 for AArch64 is used to indicate whether or not the stop has been signalled from syscall entry or syscall exit. This means that:

- Any writes by the tracer to this register during the stop are ignored/discarded.

- The actual value of the register is not available during the stop, so the tracer cannot save it and restore it later.

Right now, these registers are clobbered in tracehook_report_syscall. This change moves the logic to gpr_get and compat_gpr_get where registers are copied into a user-space buffer.

This will allow to change these registers and to introduce a new ptrace option to get the full set of registers.

---
 arch/arm64/include/asm/ptrace.h |   5 ++
 arch/arm64/kernel/ptrace.c      | 104 ++++++++++++++++++++------------
 2 files changed, 69 insertions(+), 40 deletions(-)
arch/arm64/include/asm/ptrace.h +5 -0
index e58bca832dff..0a9552b4f61e 100644
@@ -170,6 +170,11 @@ static inline unsigned long pstate_to_compat_psr(const unsigned long pstate)
 	return psr;
 }
 
+enum ptrace_syscall_dir {
+	PTRACE_SYSCALL_ENTER = 0,
+	PTRACE_SYSCALL_EXIT,
+};
+
 /*
  * This struct defines the way the registers are stored on the stack during an
  * exception. Note that sizeof(struct pt_regs) has to be a multiple of 16 (for
arch/arm64/kernel/ptrace.c +4 -1
index 8ac487c84e37..39da03104528 100644
@@ -40,6 +40,7 @@
 #include <asm/syscall.h>
 #include <asm/traps.h>
 #include <asm/system_misc.h>
+#include <asm/ptrace.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/syscalls.h>
@@ -561,7 +562,31 @@ static int gpr_get(struct task_struct *target,
 		   struct membuf to)
 {
 	struct user_pt_regs *uregs = &task_pt_regs(target)->user_regs;
-	return membuf_write(&to, uregs, sizeof(*uregs));
+	unsigned long saved_reg;
+	int ret;
+
Quoted 8 lines

arch/arm64/kernel/ptrace.c static int gpr_get(struct task_struct *target, · line 573

Quoted 6 lines
arch/arm64/kernel/ptrace.c +6 -0
+	/*
+	 * We have some ABI weirdness here in the way that we handle syscall
+	 * exit stops because we indicate whether or not the stop has been
+	 * signalled from syscall entry or syscall exit by clobbering the general
+	 * purpose register x7.
+	 */

When you move a comment, please don't truncate it!

This is my fault. In the previous version, the other part of this comment was irelevant, because I always allowed to change clobbered registers, but then I realized that we can't do that.

Quoted 15 lines

arch/arm64/kernel/ptrace.c static int gpr_get(struct task_struct *target, · line 583

Quoted 11 lines
arch/arm64/kernel/ptrace.c +10 -0
+	saved_reg = uregs->regs[7];
+
+	switch (target->ptrace_message) {
+	case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+		uregs->regs[7] = PTRACE_SYSCALL_ENTER;
+		break;
+	case PTRACE_EVENTMSG_SYSCALL_EXIT:
+		uregs->regs[7] = PTRACE_SYSCALL_EXIT;
+		break;
+	}

I'm wary of checking target->ptrace_message here, as I seem to recall the regset code also being used for coredumps. What guarantees we don't break things there?

Registers were clobbered in tracehook_report_syscall, task->ptrace_message is set in ptrace_report_syscall.

do_coredump() is called from get_signal and secure_computing, so we always see actuall registers in core dumps with and without these changes.

Quoted 113 lines
Quoted 113 lines
arch/arm64/kernel/ptrace.c +43 -33
+
+	ret =  membuf_write(&to, uregs, sizeof(*uregs));
+
+	uregs->regs[7] = saved_reg;
+
+	return ret;
 }
 
 static int gpr_set(struct task_struct *target, const struct user_regset *regset,
@@ -575,6 +600,17 @@ static int gpr_set(struct task_struct *target, const struct user_regset *regset,
 	if (ret)
 		return ret;
 
+	/*
+	 * Historically, x7 can't be changed if the stop has been signalled
+	 * from syscall-enter of syscall-exit.
+	 */
+	switch (target->ptrace_message) {
+	case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+	case PTRACE_EVENTMSG_SYSCALL_EXIT:
+		newregs.regs[7] = task_pt_regs(target)->regs[7];
+		break;
+	}
+
 	if (!valid_user_regs(&newregs, target))
 		return -EINVAL;
 
@@ -1206,6 +1242,20 @@ static inline compat_ulong_t compat_get_user_reg(struct task_struct *task, int i
 	struct pt_regs *regs = task_pt_regs(task);
 
 	switch (idx) {
+	case 12:
+		/*
+		 * We have some ABI weirdness here in the way that we handle
+		 * syscall exit stops because we indicate whether or not the
+		 * stop has been signalled from syscall entry or syscall exit
+		 * by clobbering the general purpose register r12.
+		 */
+		switch (task->ptrace_message) {
+		case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+			return PTRACE_SYSCALL_ENTER;
+		case PTRACE_EVENTMSG_SYSCALL_EXIT:
+			return PTRACE_SYSCALL_EXIT;
+		}
+		return regs->regs[idx];
 	case 15:
 		return regs->pc;
 	case 16:
@@ -1282,6 +1332,17 @@ static int compat_gpr_set(struct task_struct *target,
 
 	}
 
+	/*
+	 * Historically, x12 can't be changed if the stop has been signalled
+	 * from syscall-enter of syscall-exit.
+	 */
+	switch (target->ptrace_message) {
+	case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+	case PTRACE_EVENTMSG_SYSCALL_EXIT:
+		newregs.regs[12] = task_pt_regs(target)->regs[12];
+		break;
+	}
+
 	if (valid_user_regs(&newregs.user_regs, target))
 		*task_pt_regs(target) = newregs;
 	else
@@ -1740,53 +1801,16 @@ long arch_ptrace(struct task_struct *child, long request,
 	return ptrace_request(child, request, addr, data);
 }
 
-enum ptrace_syscall_dir {
-	PTRACE_SYSCALL_ENTER = 0,
-	PTRACE_SYSCALL_EXIT,
-};
-
 static void tracehook_report_syscall(struct pt_regs *regs,
 				     enum ptrace_syscall_dir dir)
 {
-	int regno;
-	unsigned long saved_reg;
-
-	/*
-	 * We have some ABI weirdness here in the way that we handle syscall
-	 * exit stops because we indicate whether or not the stop has been
-	 * signalled from syscall entry or syscall exit by clobbering a general
-	 * purpose register (ip/r12 for AArch32, x7 for AArch64) in the tracee
-	 * and restoring its old value after the stop. This means that:
-	 *
-	 * - Any writes by the tracer to this register during the stop are
-	 *   ignored/discarded.
-	 *
-	 * - The actual value of the register is not available during the stop,
-	 *   so the tracer cannot save it and restore it later.
-	 *
-	 * - Syscall stops behave differently to seccomp and pseudo-step traps
-	 *   (the latter do not nobble any registers).
-	 */
-	regno = (is_compat_task() ? 12 : 7);
-	saved_reg = regs->regs[regno];
-	regs->regs[regno] = dir;
-
 	if (dir == PTRACE_SYSCALL_ENTER) {
 		if (tracehook_report_syscall_entry(regs))
 			forget_syscall(regs);
-		regs->regs[regno] = saved_reg;
-	} else if (!test_thread_flag(TIF_SINGLESTEP)) {
-		tracehook_report_syscall_exit(regs, 0);
-		regs->regs[regno] = saved_reg;
 	} else {
-		regs->regs[regno] = saved_reg;
+		int singlestep = test_thread_flag(TIF_SINGLESTEP);
 
Quoted 10 lines

arch/arm64/kernel/ptrace.c long arch_ptrace(struct task_struct *child, long request, · line 1813

Quoted 7 lines
arch/arm64/kernel/ptrace.c +1 -6
-		/*
-		 * Signal a pseudo-step exception since we are stepping but
-		 * tracer modifications to the registers may have rewound the
-		 * state machine.
-		 */
-		tracehook_report_syscall_exit(regs, 1);
+		tracehook_report_syscall_exit(regs, singlestep);

Again, please preserve the comment in some form (maybe "... if we are stepping since tracer ...").

ok

Quoted 3 lines

That said, doesn't your change above break the pseudo-step trap? Currently, we report the real x7 for those.

No, it doesn't.

In case of singlestep, tracehook_report_syscall_exit calls user_single_step_report instead of ptrace_report_syscall, so current->ptace_message will not be set PTRACE_EVENTMSG_SYSCALL_EXIT.

Quoted 2 lines

Will

#5 lore raw

PATCH v2 2/3 arm64/ptrace: introduce PTRACE_O_ARM64_RAW_REGS

In reply to Andrei Vagin

We have some ABI weirdness in the way that we handle syscall exit stops because we indicate whether or not the stop has been signalled from syscall entry or syscall exit by clobbering a general purpose register (ip/r12 for AArch32, x7 for AArch64) in the tracee and restoring its old value after the stop.

This behavior was inherited from ARM and it isn't common for other architectures. Now, we have PTRACE_GET_SYSCALL_INFO that gives all required information about system calls, so the hack with clobbering registers isn't needed anymore.

This change adds the new ptrace option PTRACE_O_ARM64_RAW_REGS. If it is set, PTRACE_GETREGSET returns values of all registers without clobbering r12 or x7 and PTRACE_SETREGSE sets all registers even if a process has been stopped in syscall-enter or syscall-exit.

---
 arch/arm64/include/uapi/asm/ptrace.h |  4 ++
 arch/arm64/kernel/ptrace.c           | 70 ++++++++++++++++------------
 include/linux/ptrace.h               |  1 +
 include/uapi/linux/ptrace.h          |  9 +++-
 4 files changed, 52 insertions(+), 32 deletions(-)
arch/arm64/include/uapi/asm/ptrace.h +4 -0
index 758ae984ff97..465cc9713895 100644
@@ -109,6 +109,10 @@ struct user_hwdebug_state {
 	}		dbg_regs[16];
 };
 
+#define PTRACE_O_ARM64_RAW_REGS	(1 << 28)
+
+#define _PTRACE_O_ARCH_OPTIONS PTRACE_O_ARM64_RAW_REGS
+
 /* SVE/FP/SIMD state (NT_ARM_SVE) */
 
 struct user_sve_header {
arch/arm64/kernel/ptrace.c +39 -31

96 lines not on this page · raw message

include/linux/ptrace.h +1 -0
index 2a9df80ea887..987d6ec5f0ce 100644
@@ -46,6 +46,7 @@ extern int ptrace_access_vm(struct task_struct *tsk, unsigned long addr,
 
 #define PT_EXITKILL		(PTRACE_O_EXITKILL << PT_OPT_FLAG_SHIFT)
 #define PT_SUSPEND_SECCOMP	(PTRACE_O_SUSPEND_SECCOMP << PT_OPT_FLAG_SHIFT)
+#define PT_ARM64_RAW_REGS	(PTRACE_O_ARM64_RAW_REGS << PT_OPT_FLAG_SHIFT)
 
 /* single stepping state bits (used on ARM and PA-RISC) */
 #define PT_SINGLESTEP_BIT	31
include/uapi/linux/ptrace.h +8 -1
index 83ee45fa634b..bcc8c362ddd9 100644
@@ -7,6 +7,7 @@
 /* has the defines to get at the registers. */
 
 #include <linux/types.h>
+#include <asm/ptrace.h>
 
 #define PTRACE_TRACEME		   0
 #define PTRACE_PEEKTEXT		   1
@@ -137,8 +138,14 @@ struct ptrace_syscall_info {
 #define PTRACE_O_EXITKILL		(1 << 20)
 #define PTRACE_O_SUSPEND_SECCOMP	(1 << 21)
 
+/* (1<<28) is reserved for arch specific options. */
+#ifndef _PTRACE_O_ARCH_OPTIONS
+#define _PTRACE_O_ARCH_OPTIONS 0
+#endif
+
 #define PTRACE_O_MASK		(\
-	0x000000ff | PTRACE_O_EXITKILL | PTRACE_O_SUSPEND_SECCOMP)
+	0x000000ff | PTRACE_O_EXITKILL | \
+	PTRACE_O_SUSPEND_SECCOMP | _PTRACE_O_ARCH_OPTIONS)
 
 #include <asm/ptrace.h>
 
Signature · 1 line
2.29.2
maintainer
#6 lore raw

Re: [PATCH 2/3] arm64/ptrace: introduce PTRACE_O_ARM64_RAW_REGS

Quoted 16 lines · Andrei Vagin

We have some ABI weirdness in the way that we handle syscall exit stops because we indicate whether or not the stop has been signalled from syscall entry or syscall exit by clobbering a general purpose register (ip/r12 for AArch32, x7 for AArch64) in the tracee and restoring its old value after the stop.

This behavior was inherited from ARM and it isn't common for other architectures. Now, we have PTRACE_GET_SYSCALL_INFO that gives all required information about system calls, so the hack with clobbering registers isn't needed anymore.

This change adds the new ptrace option PTRACE_O_ARM64_RAW_REGS. If it is set, PTRACE_GETREGSET returns values of all registers without clobbering r12 or x7 and PTRACE_SETREGSE sets all registers even if a process has been stopped in syscall-enter or syscall-exit.

Quoted 7 lines
---
 arch/arm64/include/uapi/asm/ptrace.h |  4 ++
 arch/arm64/kernel/ptrace.c           | 70 ++++++++++++++++------------
 include/linux/ptrace.h               |  1 +
 include/uapi/linux/ptrace.h          |  9 +++-
 4 files changed, 52 insertions(+), 32 deletions(-)

Please split this up so that the arm64-specific changes are separate to the core changes.

Quoted 13 lines
include/uapi/linux/ptrace.h +1 -0
index 83ee45fa634b..bcc8c362ddd9 100644
@@ -7,6 +7,7 @@
 /* has the defines to get at the registers. */
 
 #include <linux/types.h>
+#include <asm/ptrace.h>
 
 #define PTRACE_TRACEME		   0
 #define PTRACE_PEEKTEXT		   1

include/uapi/linux/ptrace.h struct ptrace_syscall_info { · line 144

Quoted 7 lines
include/uapi/linux/ptrace.h +4 -0
@@ -137,8 +138,14 @@ struct ptrace_syscall_info {
 #define PTRACE_O_EXITKILL		(1 << 20)
 #define PTRACE_O_SUSPEND_SECCOMP	(1 << 21)
 
+/* (1<<28) is reserved for arch specific options. */
+#ifndef _PTRACE_O_ARCH_OPTIONS
+#define _PTRACE_O_ARCH_OPTIONS 0
+#endif

It seems a bit fragile to rely on a comment here to define the user ABI; why not define _PTRACE_O_ARCH_OPTIONS to the right value unconditionally?

Also, it seems as though we immediately burn this bit on arm64, so if we ever wanted another option we'd have to come back here and allocate another bit. Could we do better, e.g. by calling into an arch hook (arch_ptrace_setoptions()) and passing the 'addr' parameter?

How do other architectures manage this sort of thing? I'm wondering whether a separate regset containing just "real x7" and orig_x0 would be preferable after all...

Will

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#7 lore raw

Re: [PATCH 2/3] arm64/ptrace: introduce PTRACE_O_ARM64_RAW_REGS

Quoted 25 lines · Will Deacon
Quoted 16 lines · Andrei Vagin

We have some ABI weirdness in the way that we handle syscall exit stops because we indicate whether or not the stop has been signalled from syscall entry or syscall exit by clobbering a general purpose register (ip/r12 for AArch32, x7 for AArch64) in the tracee and restoring its old value after the stop.

This behavior was inherited from ARM and it isn't common for other architectures. Now, we have PTRACE_GET_SYSCALL_INFO that gives all required information about system calls, so the hack with clobbering registers isn't needed anymore.

This change adds the new ptrace option PTRACE_O_ARM64_RAW_REGS. If it is set, PTRACE_GETREGSET returns values of all registers without clobbering r12 or x7 and PTRACE_SETREGSE sets all registers even if a process has been stopped in syscall-enter or syscall-exit.

Quoted 7 lines
---
 arch/arm64/include/uapi/asm/ptrace.h |  4 ++
 arch/arm64/kernel/ptrace.c           | 70 ++++++++++++++++------------
 include/linux/ptrace.h               |  1 +
 include/uapi/linux/ptrace.h          |  9 +++-
 4 files changed, 52 insertions(+), 32 deletions(-)
Quoted 2 lines

Please split this up so that the arm64-specific changes are separate to the core changes.

ok

Quoted 17 lines
Quoted 17 lines
include/uapi/linux/ptrace.h +1 -0
index 83ee45fa634b..bcc8c362ddd9 100644
@@ -7,6 +7,7 @@
 /* has the defines to get at the registers. */
 
 #include <linux/types.h>
+#include <asm/ptrace.h>
 
 #define PTRACE_TRACEME		   0
 #define PTRACE_PEEKTEXT		   1
@@ -137,8 +138,14 @@ struct ptrace_syscall_info {
 #define PTRACE_O_EXITKILL		(1 << 20)
 #define PTRACE_O_SUSPEND_SECCOMP	(1 << 21)
 
Quoted 7 lines

include/uapi/linux/ptrace.h struct ptrace_syscall_info { · line 144

Quoted 4 lines
include/uapi/linux/ptrace.h +4 -0
+/* (1<<28) is reserved for arch specific options. */
+#ifndef _PTRACE_O_ARCH_OPTIONS
+#define _PTRACE_O_ARCH_OPTIONS 0
+#endif

It seems a bit fragile to rely on a comment here to define the user ABI; why not define _PTRACE_O_ARCH_OPTIONS to the right value unconditionally?

We don't want to allow setting options that are not supported. _PTRACE_O_ARCH_OPTIONS is added to PTRACE_O_MASK and then ptrace_setoptions checks whether all specified options is supported or not.

Quoted 5 lines

Also, it seems as though we immediately burn this bit on arm64, so if we ever wanted another option we'd have to come back here and allocate another bit. Could we do better, e.g. by calling into an arch hook (arch_ptrace_setoptions()) and passing the 'addr' parameter?

I am not sure that I understand the idea. Do you suggest to have PTRACE_O_ARCH_OPTION and pass arch-specific options in addr? In this case, I think it could be more cleaner to introduce a new ptrace command. If this is the idea, I think it worth doing this only if we expect to have more than one,two,three options.

As for my solution, we need to come back to allocate a new bit to be sure that we don't intersect with non-arch specific options. And those who add a non-arch option should see that they don't use bits of arch-specific options.

Let's decide what interface we want to use to solve the problem and then if we will stop on a ptrace option I will figure out how to improve this code.

Quoted 4 lines

How do other architectures manage this sort of thing? I'm wondering whether a separate regset containing just "real x7" and orig_x0 would be preferable after all...

Yeah, it might be a good idea. We will need to do one extra ptrace system call, but in comparison with ptrace context-switches, this is nothing.

Dave, Keno, what do you think about this?

Quoted 2 lines

Will

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#8 lore raw

PATCH v2 3/3 selftest/arm64/ptrace: add tests for PTRACE_O_ARM64_RAW_REGS

In reply to Andrei Vagin

Test output:
TAP version 13
1..2
# selftests: arm64/ptrace: ptrace_syscall_raw_regs_test # 1..2

 # ok 1 x7: 686920776f726c64
 # ok 2 The child exited with code 0.
 # # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
 ok 1 selftests: arm64/ptrace: ptrace_syscall_raw_regs_test
 # selftests: arm64/ptrace: ptrace_syscall_regs_test
 # 1..3
 # ok 1 x7: 0
 # ok 2 x7: 1
 # ok 3 The child exited with code 0.
 # # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0
 ok 2 selftests: arm64/ptrace: ptrace_syscall_regs_test
---
 tools/testing/selftests/arm64/Makefile        |   2 +-
 tools/testing/selftests/arm64/ptrace/Makefile |   6 +
 .../ptrace/ptrace_syscall_raw_regs_test.c     | 142 +++++++++++++++++
 .../arm64/ptrace/ptrace_syscall_regs_test.c   | 150 ++++++++++++++++++
 4 files changed, 299 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/arm64/ptrace/Makefile
 create mode 100644 tools/testing/selftests/arm64/ptrace/ptrace_syscall_raw_regs_test.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/ptrace_syscall_regs_test.c
tools/testing/selftests/arm64/Makefile +1 -1
index 2c9d012797a7..704770a60ece 100644
@@ -4,7 +4,7 @@
 ARCH ?= $(shell uname -m 2>/dev/null || echo not)
 
 ifneq (,$(filter $(ARCH),aarch64 arm64))
-ARM64_SUBTARGETS ?= tags signal pauth fp mte
+ARM64_SUBTARGETS ?= tags signal pauth fp mte ptrace
 else
 ARM64_SUBTARGETS :=
 endif
tools/testing/selftests/arm64/ptrace/Makefile new file +6 -0
new file mode 100644
index 000000000000..84b27449f3d1
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+CFLAGS += -g -I../../../../../usr/include/
+TEST_GEN_PROGS := ptrace_syscall_raw_regs_test ptrace_syscall_regs_test
+
+include ../../lib.mk
tools/testing/selftests/arm64/ptrace/ptrace_syscall_raw_regs_test.c new file +142 -0

142 lines not on this page · raw message

tools/testing/selftests/arm64/ptrace/ptrace_syscall_regs_test.c new file +150 -0

150 lines not on this page · raw message

Signature · 7 lines
2.29.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer
#9 lore raw

Re: [PATCH 3/3] selftest/arm64/ptrace: add tests for PTRACE_O_ARM64_RAW_REGS

[+Kees]

Quoted 18 lines · Andrei Vagin

Test output:
TAP version 13
1..2
# selftests: arm64/ptrace: ptrace_syscall_raw_regs_test # 1..2

 # ok 1 x7: 686920776f726c64
 # ok 2 The child exited with code 0.
 # # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
 ok 1 selftests: arm64/ptrace: ptrace_syscall_raw_regs_test
 # selftests: arm64/ptrace: ptrace_syscall_regs_test
 # 1..3
 # ok 1 x7: 0
 # ok 2 x7: 1
 # ok 3 The child exited with code 0.
 # # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0
 ok 2 selftests: arm64/ptrace: ptrace_syscall_regs_test
Quoted 9 lines
---
 tools/testing/selftests/arm64/Makefile        |   2 +-
 tools/testing/selftests/arm64/ptrace/Makefile |   6 +
 .../ptrace/ptrace_syscall_raw_regs_test.c     | 142 +++++++++++++++++
 .../arm64/ptrace/ptrace_syscall_regs_test.c   | 150 ++++++++++++++++++
 4 files changed, 299 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/arm64/ptrace/Makefile
 create mode 100644 tools/testing/selftests/arm64/ptrace/ptrace_syscall_raw_regs_test.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/ptrace_syscall_regs_test.c

Thanks for the tests!

We already have a pretty extensive set of syscall entry tests in tools/testing/selftests/seccomp, so perhaps this would be better off as part of that? Maybe worth a look.

Will

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#10 lore raw

Re: [PATCH 3/3] selftest/arm64/ptrace: add tests for PTRACE_O_ARM64_RAW_REGS

Quoted 31 lines · Will Deacon

[+Kees]

Quoted 18 lines · Andrei Vagin

Test output:
TAP version 13
1..2
# selftests: arm64/ptrace: ptrace_syscall_raw_regs_test # 1..2

 # ok 1 x7: 686920776f726c64
 # ok 2 The child exited with code 0.
 # # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
 ok 1 selftests: arm64/ptrace: ptrace_syscall_raw_regs_test
 # selftests: arm64/ptrace: ptrace_syscall_regs_test
 # 1..3
 # ok 1 x7: 0
 # ok 2 x7: 1
 # ok 3 The child exited with code 0.
 # # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0
 ok 2 selftests: arm64/ptrace: ptrace_syscall_regs_test
Quoted 9 lines
---
 tools/testing/selftests/arm64/Makefile        |   2 +-
 tools/testing/selftests/arm64/ptrace/Makefile |   6 +
 .../ptrace/ptrace_syscall_raw_regs_test.c     | 142 +++++++++++++++++
 .../arm64/ptrace/ptrace_syscall_regs_test.c   | 150 ++++++++++++++++++
 4 files changed, 299 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/arm64/ptrace/Makefile
 create mode 100644 tools/testing/selftests/arm64/ptrace/ptrace_syscall_raw_regs_test.c
 create mode 100644 tools/testing/selftests/arm64/ptrace/ptrace_syscall_regs_test.c
Quoted 5 lines

Thanks for the tests!

We already have a pretty extensive set of syscall entry tests in tools/testing/selftests/seccomp, so perhaps this would be better off as part of that? Maybe worth a look.

I'm happy with this living in either place -- I can make an argument either way. If it's arm64-specific, maybe better to live outside of seccomp?

Signature · 1 line
Kees Cook
#11 lore raw
In reply to Andrei Vagin

Hi Andrei,

Quoted 3 lines

This series introduces the PTRACE_O_ARM64_RAW_REGS option. If it is set, PTRACE_GETREGSET returns values of all registers, and PTRACE_SETREGSET allows to change any of them.

thanks for picking this up. I meant to work on this, but unfortunately ran out of time to be able to push it through, so I'm glad you're working on it, since it
does absolutely need to get fixed. Besides this issue, the other problem we ran into when trying to port our ptracer to aarch64 is that orig_x0 is not accessible through the ptrace interface on aarch64, which can cause tricky behavior around restarts. We managed to work around that in the end, but it's painful. If we're fixing the kernel here anyway, I'm wondering if we might want to address that as well while we're at it.

Keno

#12 lore raw
Quoted 6 lines · Keno Fischer

Hi Andrei,

Quoted 3 lines

This series introduces the PTRACE_O_ARM64_RAW_REGS option. If it is set, PTRACE_GETREGSET returns values of all registers, and PTRACE_SETREGSET allows to change any of them.

Quoted 7 lines

thanks for picking this up. I meant to work on this, but unfortunately ran out of time to be able to push it through, so I'm glad you're working on it, since it
does absolutely need to get fixed. Besides this issue, the other problem we ran into when trying to port our ptracer to aarch64 is that orig_x0 is not accessible through the ptrace interface on aarch64, which can cause tricky behavior around restarts.

Could you describe the problem in more details? I wonder whether we have the same thing in CRIU...

Quoted 3 lines

We managed to work around that in the end,
but it's painful. If we're fixing the kernel here anyway, I'm wondering if we might want to address that as well while we're at it.

Sure let think how to do this properly.

In this case, I think the ptrace option isn't a good choise. I don't think that it is a good idea to change the layout of regset depending on options...

Thanks,
Andrei

Quoted 2 lines

Keno

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#13 lore raw
Quoted 4 lines

Could you describe the problem in more details? I wonder whether we have the same thing in CRIU...

Sure, basically the issue is that orig_x0 gets recorded at the start of the syscall entry, but is not otherwise part of the ptrace state. It used to be primarily used for resetting the argument back to its original value during syscall restarts, but I see it's been expanded slightly with the user dispatch mechanism (though as far as I can tell not in a way that interacts with ptrace). Basically the problem is that if you change the value of x0 during either the ptrace entry stop or a seccomp stop and then incur a syscall restart, the syscall will restart with the original x0 rather than with the modified x0, which may be unexpected. Of course,
relatedly, if you're doing CRIU-like things you can end up
in situations where the future behavior will depend on the orig_x0 value, which isn't restore-able at the moment. It's possible to work around all of this by keeping a local copy
of orig_x0 and being very careful with the ptrace traps around restarts, but getting the logic right is extremely tricky. My suggestion for what I thought would be reasonable
behavior was:

1. Expose orig_x0 to ptrace
2. Set orig_x0 to x0 and set x0 to -ENOSYS at the start of the syscall dispatcher
3. Use orig_x0 for syscall arguments/seccomp/restarts

That's basically how rax works on x86_64 and it doesn't seem to cause major problems (though of course I may be biased by having x86_64 already work when I started the aarch64 port). Just the first item would be sufficient of course for getting rid of most of the bookkeeping. I should also say that, for us, the ptrace getregs call can be the throughput limiting operation, so it would be nice if getting the entire basic register set would only require one syscall. I won't insist on it, since we do have a solution in place that kinda works (and only requires the one syscall),
but I thought I'd mention it.

While we're on this topic, and in case it's helpful to anybody, I should also point out that the order of the ptrace-signal-stop, vs setup for the syscall restart differs between x86 and aarch64 (aarch64 sets up the restart first then delivers the ptrace trap/signal - x86 the other way around). I actually think the aarch64 behavior is saner here, but I figured I'd leave this breadcrumb for anybody who's writing a ptracer and stumbles across this.

Keno

maintainer
#14 lore raw
In reply to Andrei Vagin

Hi Andrei,

Quoted 16 lines · Andrei Vagin

Right now, ip/r12 for AArch32 and x7 for AArch64 is used to indicate whether or not the stop has been signalled from syscall entry or syscall exit. This means that:

- Any writes by the tracer to this register during the stop are ignored/discarded.

- The actual value of the register is not available during the stop, so the tracer cannot save it and restore it later.

For applications like the user-mode Linux or gVisor, it is critical to have access to the full set of registers in any moment. For example, they need to change values of all registers to emulate rt_sigreturn or execve and they need to have the full set of registers to build a signal frame.

Quoted 3 lines

This series introduces the PTRACE_O_ARM64_RAW_REGS option. If it is set, PTRACE_GETREGSET returns values of all registers, and PTRACE_SETREGSET allows to change any of them.

I haven't had a chance to go through this properly yet, but I spotted a couple of things worth mentioning off the bat:

- Please drop all of the compat changes here. The compat layer is intended to be compatible with arch/arm/, so if you want to introduce new ptrace behaviours for 32-bit applications, you need to make the changes there and then update our compat layer accordingly.

- When Keno mentioned this before [1,2], he also talked about making orig_x0 available. Since extending the ABI is a giant pain, I think this should be seriously considered.

[1] https://lore.kernel.org/r/CABV8kRzkLiVuqxT3+8c1o8m_OuROtXgfowQcrMVnrxu=CiGB=w@mail.gmail.com
[2] https://lore.kernel.org/r/CABV8kRzg1BaKdAhqXU3hONhfPAHj6Nbw0wLBC1Lo7PN1UA0CoA@mail.gmail.com

Will

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel