PowerPC: Replace kretprobe with rethook
PATCH v3 1/1 PowerPC: Replace kretprobe with rethook
This is an adaptation of commit f3a112c0c40d ("x86,rethook,kprobes:
Replace kretprobe with rethook on x86") to PowerPC.
Replaces the kretprobe code with rethook on Power. With this patch, kretprobe on Power uses the rethook instead of kretprobe specific trampoline code.
Reference to other archs:
commit b57c2f124098 ("riscv: add riscv rethook implementation") commit 7b0a096436c2 ("LoongArch: Replace kretprobe with rethook")
---
Changes in v3:
* Fixing return address moved to arch_rethook_fixup_return()
* Addressed other minor comments
* show_stack mods to show correct fn name to go in separate patch
--- arch/powerpc/Kconfig | 1 + arch/powerpc/kernel/Makefile | 1 + arch/powerpc/kernel/kprobes.c | 65 +-------------------------- arch/powerpc/kernel/optprobes.c | 2 +- arch/powerpc/kernel/rethook.c | 77 ++++++++++++++++++++++++++++++++ arch/powerpc/kernel/stacktrace.c | 6 ++- 6 files changed, 85 insertions(+), 67 deletions(-) create mode 100644 arch/powerpc/kernel/rethook.c
arch/powerpc/Kconfig +1 -0
@@ -270,6 +270,7 @@ config PPC select HAVE_PERF_EVENTS_NMI if PPC64 select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP + select HAVE_RETHOOK select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE select HAVE_RSEQ
arch/powerpc/kernel/Makefile +1 -0
@@ -140,6 +140,7 @@ obj-$(CONFIG_KPROBES) += kprobes.o obj-$(CONFIG_OPTPROBES) += optprobes.o optprobes_head.o obj-$(CONFIG_KPROBES_ON_FTRACE) += kprobes-ftrace.o obj-$(CONFIG_UPROBES) += uprobes.o +obj-$(CONFIG_RETHOOK) += rethook.o obj-$(CONFIG_PPC_UDBG_16550) += legacy_serial.o udbg_16550.o obj-$(CONFIG_SWIOTLB) += dma-swiotlb.o obj-$(CONFIG_ARCH_HAS_DMA_SET_MASK) += dma-mask.o
arch/powerpc/kernel/kprobes.c +1 -64
85 lines not on this page · raw message
arch/powerpc/kernel/optprobes.c +1 -1
@@ -56,7 +56,7 @@ static unsigned long can_optimize(struct kprobe *p)* has a 'nop' instruction, which can be emulated. * So further checks can be skipped. */ -if (p->addr == (kprobe_opcode_t *)&__kretprobe_trampoline) +if (p->addr == (kprobe_opcode_t *)&arch_rethook_trampoline)return addr +sizeof (kprobe_opcode_t);/*
arch/powerpc/kernel/rethook.c new file +77 -0
77 lines not on this page · raw message
arch/powerpc/kernel/stacktrace.c +4 -2
@@ -21,6 +21,7 @@#include <asm/processor.h> #include <linux/ftrace.h> #include <asm/kprobes.h> +#include <linux/rethook.h> #include <asm/paca.h>
@@ -133,12 +134,13 @@ int __no_sanitize_address arch_stack_walk_reliable(stack_trace_consume_fn consum* arch-dependent code, they are generic. */ ip =ftrace_graph_ret_addr (task, &graph_idx, ip, stack); -#ifdef CONFIG_KPROBES +/* * Mark stacktraces with kretprobed functions on them * as unreliable. */ -if (ip == (unsigned long )__kretprobe_trampoline) +#ifdef CONFIG_RETHOOK +if (ip == (unsigned long )arch_rethook_trampoline)return -EINVAL ;#endif
Signature · 1 line
2.44.0
Hi Abhishek,
This patch looks good to me, except for one main change in trampoline_rethook_handler().
Other than that, I have a few more minor comments. Sorry, I should have caught that last time. But, the other change needs a new submission, so good to get these addressed as well.
For the subject, we generally use 'powerpc' (all lowercase):
powerpc: Replace kretprobe with rethook
Quoted 2 lines · Abhishek Dubey
This is an adaptation of commit
f3a112c0c40d("x86,rethook,kprobes:
Replace kretprobe with rethook on x86") to PowerPC.
^^^^^^^
Here too.
Quoted 4 lines
Replaces the kretprobe code with rethook on Power. With this patch, kretprobe on Power uses the rethook instead of kretprobe specific trampoline code.
In general, it is good to describe what is being done in the patch. So, rather than 'Replaces', you can start that with 'Replace kretprobe code with rethook on powerpc.'
The second sentence also looks redundant. It is good if you can describe what the patch does where possible. For instance:
Rethook follows the existing kretprobe implementation, but separates it from kprobes so that it can be used by fprobe (ftrace-based function entry/exit probes). As such, this patch also enables fprobe to work on powerpc. The only other change compared to the existing kretprobe implementation is doing the return address fixup in arch_rethook_fixup_return().
Quoted 180 lines
Reference to other archs:
commitb57c2f124098("riscv: add riscv rethook implementation") commit7b0a096436c2("LoongArch: Replace kretprobe with rethook")Signed-off-by Abhishek Dubey <[email protected]>---
Changes in v3:
* Fixing return address moved toarch_rethook_fixup_return()
* Addressed other minor comments
*show_stackmods to show correct fn name to go in separate patch--- arch/powerpc/Kconfig | 1 + arch/powerpc/kernel/Makefile | 1 + arch/powerpc/kernel/kprobes.c | 65 +-------------------------- arch/powerpc/kernel/optprobes.c | 2 +- arch/powerpc/kernel/rethook.c | 77 ++++++++++++++++++++++++++++++++ arch/powerpc/kernel/stacktrace.c | 6 ++- 6 files changed, 85 insertions(+), 67 deletions(-) create mode 100644 arch/powerpc/kernel/rethook.carch/powerpc/Kconfig +1
-0@@ -270,6 +270,7 @@ config PPC select HAVE_PERF_EVENTS_NMI if PPC64 select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP + select HAVE_RETHOOK select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE select HAVE_RSEQarch/powerpc/kernel/Makefile +1
-0@@ -140,6 +140,7 @@ obj-$(CONFIG_KPROBES) += kprobes.o obj-$(CONFIG_OPTPROBES) += optprobes.o optprobes_head.o obj-$(CONFIG_KPROBES_ON_FTRACE) += kprobes-ftrace.o obj-$(CONFIG_UPROBES) += uprobes.o +obj-$(CONFIG_RETHOOK) += rethook.o obj-$(CONFIG_PPC_UDBG_16550) += legacy_serial.o udbg_16550.o obj-$(CONFIG_SWIOTLB) += dma-swiotlb.o obj-$(CONFIG_ARCH_HAS_DMA_SET_MASK) += dma-mask.oarch/powerpc/kernel/kprobes.c +1
-64@@ -228,16 +228,6 @@ static nokprobe_inline void set_current_kprobe(struct kprobe *p, struct pt_regs kcb->kprobe_saved_msr = regs->msr; } -void arch_prepare_kretprobe (struct kretprobe_instance *ri,struct pt_regs *regs) -{ - ri->ret_addr = (kprobe_opcode_t *)regs->link; - ri->fp =NULL ; - -/* Replace the return addr with trampoline addr */ - regs->link = (unsigned long )__kretprobe_trampoline; -} -NOKPROBE_SYMBOL (arch_prepare_kretprobe); -static int try_to_emulate (struct kprobe *p,struct pt_regs *regs) {int ret;@@ -394,49 +384,6 @@ int kprobe_handler(struct pt_regs *regs) }NOKPROBE_SYMBOL (kprobe_handler); -/* -* Function return probe trampoline: -* - init_kprobes() establishes a probepoint here -* - When the probed function returns, this probe -* causes the handlers to fire -*/ -asm(".global __kretprobe_trampoline\n" -".type __kretprobe_trampoline, @function\n" -"__kretprobe_trampoline:\n" -"nop\n" -"blr\n" -".size __kretprobe_trampoline, .-__kretprobe_trampoline\n" ); - -/* -* Called when the probe at kretprobe trampoline is hit -*/ -static int trampoline_probe_handler (struct kprobe *p,struct pt_regs *regs) -{ -unsigned long orig_ret_address; - - orig_ret_address =__kretprobe_trampoline_handler (regs,NULL ); -/* -* We get here through one of two paths: -* 1. by taking a trap -> kprobe_handler() -> here -* 2. by optprobe branch -> optimized_callback() -> opt_pre_handler() -> here -* -* When going back through (1), we need regs->nip to be setup properly -* as it is used to determine the return address from the trap. -* For (2), since nip is not honoured with optprobes, we instead setup -* the link register properly so that the subsequent 'blr' in -* __kretprobe_trampoline jumps back to the right instruction. -* -* For nip, we should set the address to the previous instruction since -* we end up emulating it in kprobe_handler(), which increments the nip -* again. -*/ -regs_set_return_ip (regs, orig_ret_address -4 ); - regs->link = orig_ret_address; - -return 0 ; -} -NOKPROBE_SYMBOL (trampoline_probe_handler); -/* * Called after single-stepping. p->addr is the address of the * instruction whose first byte has been replaced by the "breakpoint" @@ -539,19 +486,9 @@ int kprobe_fault_handler(struct pt_regs *regs, int trapnr) }NOKPROBE_SYMBOL (kprobe_fault_handler); -static struct kprobe trampoline_p = { - .addr = (kprobe_opcode_t *) &__kretprobe_trampoline, - .pre_handler = trampoline_probe_handler -}; - -int __init arch_init_kprobes (void ) -{ -return register_kprobe (&trampoline_p); -} -int arch_trampoline_kprobe (struct kprobe *p) { -if (p->addr == (kprobe_opcode_t *)&__kretprobe_trampoline) +if (p->addr == (kprobe_opcode_t *)&arch_rethook_trampoline)return 1 ;return 0 ;arch/powerpc/kernel/optprobes.c +1
-1@@ -56,7 +56,7 @@ static unsigned long can_optimize(struct kprobe *p)* has a 'nop' instruction, which can be emulated. * So further checks can be skipped. */ -if (p->addr == (kprobe_opcode_t *)&__kretprobe_trampoline) +if (p->addr == (kprobe_opcode_t *)&arch_rethook_trampoline)return addr +sizeof (kprobe_opcode_t);/* arch/powerpc/kernel/rethook.c new file +22
-0@@ -0,0 +1,77 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* PowerPC implementation of rethook. This depends on kprobes. +*/ + +#include <linux/kprobes.h> +#include <linux/rethook.h> + +/* +* Function return trampoline: +* - init_kprobes() establishes a probepoint here +* - When the probed function returns, this probe +* causes the handlers to fire +*/ +asm(".global arch_rethook_trampoline\n" +".type arch_rethook_trampoline, @function\n" +"arch_rethook_trampoline:\n" +"nop\n" +"blr\n" +".size arch_rethook_trampoline, .-arch_rethook_trampoline\n" ); + +/*
arch/powerpc/kernel/rethook.c line 32
Quoted 10 lines
arch/powerpc/kernel/rethook.c new file +10
-0+* Called when the probe at kretprobe trampoline is hit +*/ +static int trampoline_rethook_handler (struct kprobe *p,struct pt_regs *regs) +{ +unsigned long orig_ret_address; + + orig_ret_address =rethook_trampoline_handler (regs,0 ); +rethook_trampoline_handler (regs, orig_ret_address); +return 0 ; +}
I think we should pass in regs->gpr[1] to allow it to be verified against the stack pointer we saved on function entry. This can be simplified to:
static int trampoline_rethook_handler(struct kprobe *p, struct pt_regs *regs)
{
return !rethook_trampoline_handler(regs, regs->gpr[1]);
}I have tested this patch with that change. So, with that change included:
Quoted 15 lines
arch/powerpc/kernel/rethook.c new file +15
-0+NOKPROBE_SYMBOL (trampoline_rethook_handler); + +void arch_rethook_prepare (struct rethook_node *rh,struct pt_regs *regs,bool mcount) +{ + rh->ret_addr = regs->link; + rh->frame = regs->gpr[1 ]; + +/* Replace the return addr with trampoline addr */ + regs->link = (unsigned long )arch_rethook_trampoline; +} +NOKPROBE_SYMBOL (arch_rethook_prepare); + +/* This is called from rethook_trampoline_handler(). */ +void arch_rethook_fixup_return (struct pt_regs *regs,unsigned long orig_ret_address) +{
arch/powerpc/kernel/rethook.c line 57
Quoted 10 lines
arch/powerpc/kernel/rethook.c new file +10
-0+/* +* We get here through one of two paths: +* 1. by taking a trap -> kprobe_handler() -> here +* 2. by optprobe branch -> optimized_callback() -> opt_pre_handler() -> here +* +* When going back through (1), we need regs->nip to be setup properly +* as it is used to determine the return address from the trap. +* For (2), since nip is not honoured with optprobes, we instead setup +* the link register properly so that the subsequent 'blr' in +* __kretprobe_trampoline jumps back to the right instruction.
^^^^^^^^^^^^^^^^^^^^^^^ arch_rethook_trampoline
Thanks!
- Naveen
Quoted 51 lines
arch/powerpc/kernel/rethook.c new file +20
-0+* +* For nip, we should set the address to the previous instruction since +* we end up emulating it in kprobe_handler(), which increments the nip +* again. +*/ +regs_set_return_ip (regs, orig_ret_address -4 ); + regs->link = orig_ret_address; +} +NOKPROBE_SYMBOL (arch_rethook_fixup_return); + +static struct kprobe trampoline_p = { + .addr = (kprobe_opcode_t *) &arch_rethook_trampoline, + .pre_handler = trampoline_rethook_handler +}; + +/* rethook initializer */ +int __init arch_init_kprobes (void ) +{ +return register_kprobe (&trampoline_p); +}arch/powerpc/kernel/stacktrace.c +4
-2@@ -21,6 +21,7 @@#include <asm/processor.h> #include <linux/ftrace.h> #include <asm/kprobes.h> +#include <linux/rethook.h> #include <asm/paca.h> @@ -133,12 +134,13 @@ int __no_sanitize_address arch_stack_walk_reliable(stack_trace_consume_fn consum* arch-dependent code, they are generic. */ ip =ftrace_graph_ret_addr (task, &graph_idx, ip, stack); -#ifdef CONFIG_KPROBES +/* * Mark stacktraces with kretprobed functions on them * as unreliable. */ -if (ip == (unsigned long )__kretprobe_trampoline) +#ifdef CONFIG_RETHOOK +if (ip == (unsigned long )arch_rethook_trampoline)return -EINVAL ;#endif Signature · 1 line
2.44.0
arch/powerpc/kernel/rethook.c line 77
Quoted 5 lines · On Thu, 27 Jun 2024 09:21:01 -0400 Abhishek Dubey <[email protected]> wrote:
arch/powerpc/kernel/rethook.c new file +5
-0+/* rethook initializer */ +int __init arch_init_kprobes (void ) +{ +return register_kprobe (&trampoline_p); +}
No, please don't use arch_init_kprobes() for initializing rethook, since rethook is used from fprobe too (at this moment).
If you want to make it relays on kprobes, you have to make a dependency in powerpc's kconfig, e.g.
select HAVE_RETHOOK if KPROBES
But I don't recommend it.
Thank you,
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
Quoted 12 lines · Masami Hiramatsu
arch/powerpc/kernel/rethook.c line 77
Quoted 5 lines · On Thu, 27 Jun 2024 09:21:01 -0400 Abhishek Dubey <[email protected]> wrote:
arch/powerpc/kernel/rethook.c new file +5
-0+/* rethook initializer */ +int __init arch_init_kprobes (void ) +{ +return register_kprobe (&trampoline_p); +}No, please don't use
arch_init_kprobes()for initializing rethook, since rethook is used from fprobe too (at this moment).
Quoted 6 lines
If you want to make it relays on kprobes, you have to make a dependency in powerpc's kconfig, e.g.
select
HAVE_RETHOOKif KPROBESBut I don't recommend it.
Given that kretprobe has always worked this way on powerpc, I think this is a fair tradeoff. We get to enable fprobes on powerpc only if kprobes is also enabled.
Longer term, it would certainly be nice to get rid of that probe, and to expand the trampoline to directly invoke the rethook callback.
Thanks,
Naveen
Quoted 18 lines · On Tue, 09 Jul 2024 12:28:29 +0530 Naveen N Rao <[email protected]> wrote:
Quoted 17 lines · Masami Hiramatsu
arch/powerpc/kernel/rethook.c line 77
Quoted 5 lines · On Thu, 27 Jun 2024 09:21:01 -0400 Abhishek Dubey <[email protected]> wrote:
arch/powerpc/kernel/rethook.c new file +5
-0+/* rethook initializer */ +int __init arch_init_kprobes (void ) +{ +return register_kprobe (&trampoline_p); +}No, please don't use
arch_init_kprobes()for initializing rethook, since rethook is used from fprobe too (at this moment).If you want to make it relays on kprobes, you have to make a dependency in powerpc's kconfig, e.g.
select
HAVE_RETHOOKif KPROBES
Quoted 8 lines
Quoted 1 line
But I don't recommend it.
Given that kretprobe has always worked this way on powerpc, I think this is a fair tradeoff. We get to enable fprobes on powerpc only if kprobes is also enabled.
Longer term, it would certainly be nice to get rid of that probe, and to expand the trampoline to directly invoke the rethook callback.
OK. In longer term, rethook will be only for kretprobe, and kretprobe will be replaced by fprobe[1]. So please comment it and add that
[1] https://lore.kernel.org/all/172000134410.63468.13742222887213469474.stgit@devnote2/
Thank you,
Quoted 4 lines
Thanks,
Naveen
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>