x86: ptrace: Add function argument access API

A while ago I posted an RFC patchset for dynamic function based events. But Masami pointed out that this could be done with kprobes with minimal changes. He posted a patch set back in March
http://lkml.kernel.org/r/152049860385.7289.14079393589900496424.stgit@devbox
I've pulled this in locally, but haven't had the time until recently to look at it seriously. I even plan to talk about these changes in my talk at Open Source Summit in Edinburgh less than two weeks away (talk about conference driven development!).
Anyway, the one patch that really needs external approval is the one that creates a new architecture dependent API to retrieve function arguments from pt_regs if the ip is at the start of the function call (via a breakpoint or ftrace fentry). That's this patch.

Anyone have any issues with it? If not, I'm going to start doing some serious testing of this code and try to get it into the next merge window.

[
A while ago I posted an RFC patchset for dynamic function based events. But Masami pointed out that this could be done with kprobes with minimal changes. He posted a patch set back in March
http://lkml.kernel.org/r/152049860385.7289.14079393589900496424.stgit@devbox
I've pulled this in locally, but haven't had the time until recently to look at it seriously. I even plan to talk about these changes in my talk at Open Source Summit in Edinburgh less than two weeks away (talk about conference driven development!).
Anyway, the one patch that really needs external approval is the one that creates a new architecture dependent API to retrieve function arguments from pt_regs if the ip is at the start of the function call (via a breakpoint or ftrace fentry). That's this patch.

Anyone have any issues with it? If not, I'm going to start doing some serious testing of this code and try to get it into the next merge window.

Thanks!

-- Steve
]

From: Masami Hiramatsu <[email protected]>

Add regs_get_argument() which returns N th argument of the function call.
Note that this chooses most probably assignment, in some case it can be incorrect (e.g. passing data structure or floating point etc.)

This is expected to be called from kprobes or ftrace with regs where the top of stack is the return address.

Link http://lkml.kernel.org/r/152465885737.26224.2822487520472783854.stgit@devbox open here

Signed-off-by Masami Hiramatsu <[email protected]>

Signed-off-by Steven Rostedt (VMware) <[email protected]>

---
 arch/Kconfig                  |  7 +++++++
 arch/x86/Kconfig              |  1 +
 arch/x86/include/asm/ptrace.h | 38 +++++++++++++++++++++++++++++++++++
 3 files changed, 46 insertions(+)

arch/Kconfig +7 -0

index 6801123932a5..facace0c90fc 100644
@@ -290,6 +290,13 @@ config HAVE_RSEQ
 	  This symbol should be selected by an architecture if it
 	  supports an implementation of restartable sequences.
 
+config HAVE_FUNCTION_ARG_ACCESS_API
+	bool
+	help
+	  This symbol should be selected by an architecure if it supports
+	  the API needed to access function arguments from pt_regs,
+	  declared in asm/ptrace.h
+
 config HAVE_CLK
 	bool
 	help

arch/x86/Kconfig +1 -0

index 1a0be022f91d..972973851779 100644
@@ -184,6 +184,7 @@ config X86
 	select HAVE_RCU_TABLE_INVALIDATE	if HAVE_RCU_TABLE_FREE
 	select HAVE_REGS_AND_STACK_ACCESS_API
 	select HAVE_RELIABLE_STACKTRACE		if X86_64 && (UNWINDER_FRAME_POINTER || UNWINDER_ORC) && STACK_VALIDATION
+	select HAVE_FUNCTION_ARG_ACCESS_API
 	select HAVE_STACKPROTECTOR		if CC_HAS_SANE_STACKPROTECTOR
 	select HAVE_STACK_VALIDATION		if X86_64
 	select HAVE_RSEQ

arch/x86/include/asm/ptrace.h +38 -0

index 6de1fd3d0097..c2304b25e2fd 100644
@@ -256,6 +256,44 @@ static inline unsigned long regs_get_kernel_stack_nth(struct pt_regs *regs,
 		return 0;
 }
 
+/**
+ * regs_get_kernel_argument() - get Nth function argument in kernel
+ * @regs:	pt_regs of that context
+ * @n:		function argument number (start from 0)
+ *
+ * regs_get_argument() returns @n th argument of the function call.
+ * Note that this chooses most probably assignment, in some case
+ * it can be incorrect.
+ * This is expected to be called from kprobes or ftrace with regs
+ * where the top of stack is the return address.
+ */
+static inline unsigned long regs_get_kernel_argument(struct pt_regs *regs,
+						     unsigned int n)
+{
+	static const unsigned int argument_offs[] = {
+#ifdef __i386__
+		offsetof(struct pt_regs, ax),
+		offsetof(struct pt_regs, cx),
+		offsetof(struct pt_regs, dx),
+#define NR_REG_ARGUMENTS 3
+#else
+		offsetof(struct pt_regs, di),
+		offsetof(struct pt_regs, si),
+		offsetof(struct pt_regs, dx),
+		offsetof(struct pt_regs, cx),
+		offsetof(struct pt_regs, r8),
+		offsetof(struct pt_regs, r9),
+#define NR_REG_ARGUMENTS 6
+#endif
+	};
+
+	if (n >= NR_REG_ARGUMENTS) {
+		n -= NR_REG_ARGUMENTS - 1;
+		return regs_get_kernel_stack_nth(regs, n);
+	} else
+		return regs_get_register(regs, argument_offs[n]);
+}
+
 #define arch_has_single_step()	(1)
 #ifdef CONFIG_X86_DEBUGCTLMSR
 #define arch_has_block_step()	(1)

Anyone have any issues with this patch?

Quoted 4 lines · Steven Rostedt

Anyone have any issues with this patch?

I'm conceptually okay with it. That being said, regs_within_kernel_stack(), which you're indirectly using, is off-by-a-few. And updating it to use probe_kernel_read() might be nice for robustness.

Quoted 4 lines · Steven Rostedt

Anyone have any issues with this patch?

I'm conceptually okay with it. That being said, regs_within_kernel_stack(), which you're indirectly using, is off-by-a-few. And updating it to use probe_kernel_read() might be nice for robustness.

Quoted 11 lines · On Fri, 12 Oct 2018 11:21:28 -0700 Andy Lutomirski <[email protected]> wrote:

Quoted 4 lines · Steven Rostedt

Anyone have any issues with this patch?

I'm conceptually okay with it. That being said, regs_within_kernel_stack(), which you're indirectly using, is off-by-a-few. And updating it to use probe_kernel_read() might be nice for robustness.

Something like this?

-- Steve

From: "Steven Rostedt (VMware)" <[email protected]>
Date: Fri, 12 Oct 2018 15:44:20 -0400
Subject: [PATCH] x86: ptrace.h: Add regs_get_kernel_stack_nth_safe() function

Andy had some concerns about using regs_get_kernel_stack_nth() in a new function regs_get_kernel_argument() as if there's any error in the stack code, it could cause a bad memory access. Instead, add a new function called regs_get_kernel_stack_nth_safe() that does a probe_kernel_read() on the stack address to be extra careful in accessing the memory. To share the code, regs_get_kernel_stack_nth_addr() was added to just return the stack address (or NULL if not on the stack), that both regs_get_kernel_stack_nth() and the _safe() version can use.

Link http://lkml.kernel.org/r/CALCETrXn9zKTb9i1LP3qoFcpqZHF34BdkuZ5D3N0uCmRr+VnbA@mail.gmail.com open here

Requested-by Andy Lutomirski <[email protected]>

Signed-off-by Steven Rostedt (VMware) <[email protected]>

---
 arch/x86/include/asm/ptrace.h | 57 ++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 54 insertions(+), 3 deletions(-)

arch/x86/include/asm/ptrace.h +54 -3

index c2304b25e2fd..8df7ab6a17c5 100644
@@ -237,6 +237,27 @@ static inline int regs_within_kernel_stack(struct pt_regs *regs,
 }
 
 /**
+ * regs_get_kernel_stack_nth_addr() - get the address of the Nth entry on stack
+ * @regs:	pt_regs which contains kernel stack pointer.
+ * @n:		stack entry number.
+ *
+ * regs_get_kernel_stack_nth() returns the address of the @n th entry of the
+ * kernel stack which is specified by @regs. If the @n th entry is NOT in
+ * the kernel stack, this returns NULL.
+ */
+static inline unsigned long *regs_get_kernel_stack_nth_addr(struct pt_regs *regs,
+							    unsigned int n)
+{
+	unsigned long *addr = (unsigned long *)kernel_stack_pointer(regs);
+
+	addr += n;
+	if (regs_within_kernel_stack(regs, (unsigned long)addr))
+		return addr;
+	else
+		return NULL;
+}
+
+/**
  * regs_get_kernel_stack_nth() - get Nth entry of the stack
  * @regs:	pt_regs which contains kernel stack pointer.
  * @n:		stack entry number.

@@ -248,14 +269,44 @@ static inline int regs_within_kernel_stack(struct pt_regs *regs,
 static inline unsigned long regs_get_kernel_stack_nth(struct pt_regs *regs,
 						      unsigned int n)
 {
-	unsigned long *addr = (unsigned long *)kernel_stack_pointer(regs);
-	addr += n;
-	if (regs_within_kernel_stack(regs, (unsigned long)addr))
+	unsigned long *addr;
+
+	addr = regs_get_kernel_stack_nth_addr(regs, n);
+	if (addr)
 		return *addr;
 	else
 		return 0;
 }
 
+/* To avoid include hell, we can't include uaccess.h */
+extern long probe_kernel_read(void *dst, const void *src, size_t size);
+
+/**
+ * regs_get_kernel_stack_nth_safe() - safely get Nth entry of the stack
+ * @regs:	pt_regs which contains kernel stack pointer.
+ * @n:		stack entry number.
+ *
+ * Same as regs_get_kernel_stack_nth(), but references the stack value
+ * with a probe_kernel_read() in case there's a bad stack pointer, it
+ * will not cause a bad memory access. If the @n is not on the stack,
+ * or a bad memory access happened, it returns zero.
+ */
+static inline unsigned long regs_get_kernel_stack_nth_safe(struct pt_regs *regs,
+							   unsigned int n)
+{
+	unsigned long *addr;
+	unsigned long val;
+	long ret;
+
+	addr = regs_get_kernel_stack_nth_addr(regs, n);
+	if (addr) {
+		ret = probe_kernel_read(&val, addr, sizeof(val));
+		if (!ret)
+			return val;
+	}
+	return 0;
+}
+
 /**
  * regs_get_kernel_argument() - get Nth function argument in kernel
  * @regs:	pt_regs of that context

Signature · 1 line

2.13.6

This patch looks good to me.
But if the concern is real, all regs_get_kernel_stack_nth() user must move onto _safe() version, at least all tracers code.

Reviewed-by Masami Hiramatsu <[email protected]>

Quoted 23 lines · On Fri, 12 Oct 2018 15:54:39 -0400 Steven Rostedt <[email protected]> wrote:

Quoted 11 lines · On Fri, 12 Oct 2018 11:21:28 -0700 Andy Lutomirski <[email protected]> wrote:

Quoted 4 lines · Steven Rostedt

Anyone have any issues with this patch?

I'm conceptually okay with it. That being said, regs_within_kernel_stack(), which you're indirectly using, is off-by-a-few. And updating it to use probe_kernel_read() might be nice for robustness.

Something like this?

-- Steve

From: "Steven Rostedt (VMware)" <[email protected]>
Date: Fri, 12 Oct 2018 15:44:20 -0400
Subject: [PATCH] x86: ptrace.h: Add regs_get_kernel_stack_nth_safe() function

Quoted 8 lines

Andy had some concerns about using regs_get_kernel_stack_nth() in a new function regs_get_kernel_argument() as if there's any error in the stack code, it could cause a bad memory access. Instead, add a new function called regs_get_kernel_stack_nth_safe() that does a probe_kernel_read() on the stack address to be extra careful in accessing the memory. To share the code, regs_get_kernel_stack_nth_addr() was added to just return the stack address (or NULL if not on the stack), that both regs_get_kernel_stack_nth() and the _safe() version can use.

This patch looks good to me.
But if the concern is real, all regs_get_kernel_stack_nth() user must move onto _safe() version, at least all tracers code.

Reviewed-by Masami Hiramatsu <[email protected]>

Quoted 4 lines · Steven Rostedt

Anyone have any issues with this patch?

I'm conceptually okay with it. That being said, regs_within_kernel_stack(), which you're indirectly using, is off-by-a-few. And updating it to use probe_kernel_read() might be nice for robustness.