kprobes: Cleanup jprobe implementation
Subsystems (15)
Masami Hiramatsu
maintainerv4 cover kprobes: Cleanup jprobe implementation
Hello,
Since we decided to remove jprobe from kernel last year, its APIs are disabled and we worked on moving in-kernel jprobe users to kprobes or trace-events. And now no jprobe users are here anymore.
This is the 4th version of the series for removing jprobe. Previous version is here:
https://lkml.org/lkml/2018/5/16/1052
Unlike previous versions, this version removes jprobe from all architectures, as Ingo asked me in the previous thread.
I marked this as RFC again because it introduced changes for many arch. It should be reviewed by some arch which is not simply removing code (e.g. powerpc, arm).
BTW, this series are based on -tip tree as same as its previous version, and I'm sure it can be applied to linus tree/linux-next too.
Series structure
=====
Basically this series introduces 3 major changes;
- Remove jprobe implementation ([2/27] - [12/27])
- Remove break_handler() related code ([13/27] - [23/27]) This should be done after (or with) removing jprobe because break_handler is only used by jprobe. (I'm not so sure, should I merge above patches into one arch-wide patch as next one does?)
- Clean current_kprobe and enable preempt if pre_handler() returns !0 ([24/27] and [25/27]<- x86 specific update) This also depends on above patches because those current_kprobe and preemption are expected to be adjusted by jprobe implementation via break_handler.
And some minor changes;
- Document cleanup and update ([1/27], [26/27])
And finally remove jprobe stub APIs and break_handler from kprobes.h ([27/27]). Of course this depends on above patches.
Since removing jprobes and break_handler related code involve archtecture specific changes in some archs, I splitted it for each arch. But "clean current_kprobe and enable preempt if pre_handler() returns !0" patch modifies all architectures, since it changes expected kprobe handler behavior. I think this would be better done in one patch for consistency.
Result
=====
I've tested it with kprobe sanity test on x86-64, and arm64 but for other archs, I just did cross-build test.
With this series, we finally cleanup all jprobe code and break_handler as below.
$ git grep -wi break_handler | wc -l 0 $ git grep -wi jprobe | wc -l 0
Actually, there is one place where mentioning jprobe, Documentation/kprobes.txt explains jprobe is deprecated and how to migrate jprobe user to ftrace or kprobe. I didn't remove it because it might be useful for some users.
Thank you,
---
Masami Hiramatsu (27):
Documentation/kprobes: Fix to remove remaining jprobe
kprobes: Remove jprobe API implementation
kprobes/x86: Remove jprobe implementation
ARC: kprobes: Remove jprobe implementation
ARM: kprobes: Remove jprobe arm implementation
arm64: kprobes: Remove jprobe implementation
powerpc/kprobes: Remove jprobe powerpc implementation
ia64: kprobes: Remove jprobe implementation
MIPS: kprobes: Remove jprobe implementation
s390/kprobes: Remove jprobe implementation
sh: kprobes: Remove jprobe implementation
sparc64: kprobes: Remove jprobe implementation
kprobes: Don't check the ->break_handler() in generic kprobes code kprobes/x86: Don't call ->break_handler() in x86 kprobes
ARC: kprobes: Don't call the ->break_handler() in ARC kprobes code
ARM: kprobes: Don't call the ->break_handler() in arm kprobes code
arm64: kprobes: Don't call the ->break_handler() in arm kprobes code powerpc/kprobes: Don't call the ->break_handler() in arm kprobes code
ia64: kprobes: Don't call the ->break_handler() in ia64 kprobes code
MIPS: kprobes: Don't call the ->break_handler() in MIPS kprobes code s390/kprobes: Don't call the ->break_handler() in s390 kprobes code
sh: kprobes: Don't call the ->break_handler() in SH kprobes code
sparc64: kprobes: Don't call the ->break_handler() in sparc64 kprobes code
bpf: error-inject: kprobes: Clear current_kprobe and enable preempt in kprobe
x86: kprobes: Do not disable preempt on int3 path
Documentation: kprobes: Add how to change the execution path
kprobes: Remove jprobe stub API
Documentation/kprobes.txt | 35 +++++- arch/arc/include/asm/kprobes.h | 2 arch/arc/kernel/kprobes.c | 50 +-------- arch/arm/include/asm/kprobes.h | 2 arch/arm/include/asm/probes.h | 1 arch/arm/probes/kprobes/core.c | 135 +----------------------- arch/arm64/include/asm/kprobes.h | 1 arch/arm64/kernel/probes/kprobes.c | 86 +-------------- arch/ia64/include/asm/kprobes.h | 2 arch/ia64/include/uapi/asm/break.h | 1 arch/ia64/kernel/Makefile | 2 arch/ia64/kernel/jprobes.S | 90 ---------------- arch/ia64/kernel/kprobes.c | 93 +---------------- arch/mips/include/asm/kprobes.h | 13 -- arch/mips/kernel/kprobes.c | 70 ++---------- arch/powerpc/include/asm/kprobes.h | 12 -- arch/powerpc/kernel/kprobes-ftrace.c | 31 +----- arch/powerpc/kernel/kprobes.c | 92 +++------------- arch/powerpc/kernel/trace/ftrace_64_mprofile.S | 39 +------ arch/s390/include/asm/kprobes.h | 2 arch/s390/kernel/kprobes.c | 86 +-------------- arch/sh/include/asm/kprobes.h | 4 - arch/sh/kernel/kprobes.c | 72 +------------ arch/sparc/include/asm/kprobes.h | 1 arch/sparc/kernel/kprobes.c | 65 +----------- arch/x86/include/asm/kprobes.h | 5 - arch/x86/kernel/kprobes/common.h | 10 -- arch/x86/kernel/kprobes/core.c | 119 +-------------------- arch/x86/kernel/kprobes/ftrace.c | 31 ++---- arch/x86/kernel/kprobes/opt.c | 1 include/linux/kprobes.h | 53 --------- kernel/fail_function.c | 3 - kernel/kprobes.c | 115 +------------------- kernel/test_kprobes.c | 94 ----------------- kernel/trace/trace_kprobe.c | 11 +- lib/Kconfig.debug | 2 36 files changed, 125 insertions(+), 1306 deletions(-) delete mode 100644 arch/ia64/kernel/jprobes.S
--
Masami Hiramatsu (Linaro)
Masami Hiramatsu
maintainerPATCH v4 1/27 Documentation/kprobes: Fix to remove remaining jprobe
Remove jps from the document, since jprobe is removed.
--- Documentation/kprobes.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/kprobes.txt +1 -1
@@ -474,7 +474,7 @@ error occurs during registration, all probes in the array, up to the bad probe, are safely unregistered before the register_*probes function returns. -- kps/rps/jps: an array of pointers to ``*probe`` data structures +- kps/rps: an array of pointers to ``*probe`` data structures - num: the number of the array entries. .. note::
Masami Hiramatsu
maintainerPATCH v4 2/27 kprobes: Remove jprobe API implementation
In reply to Masami HiramatsuRemove jprobe API implementations and test cases for those APIs which is no more used.
---
Changes in v4:
- Cleanup lib/Kconfig.debug description too.
Changes in v3:
- Remove test cases.
--- include/linux/kprobes.h | 3 -- kernel/kprobes.c | 78 +-------------------------------------- kernel/test_kprobes.c | 94 ----------------------------------------------- lib/Kconfig.debug | 2 + 4 files changed, 2 insertions(+), 175 deletions(-)
include/linux/kprobes.h +0 -3
@@ -389,9 +389,6 @@ int register_kprobe(struct kprobe *p);void unregister_kprobe (struct kprobe *p);int register_kprobes (struct kprobe **kps,int num);void unregister_kprobes (struct kprobe **kps,int num); -int setjmp_pre_handler (struct kprobe *,struct pt_regs *); -int longjmp_break_handler (struct kprobe *,struct pt_regs *); -void jprobe_return (void );unsigned long arch_deref_entry_point (void *);int register_kretprobe (struct kretprobe *rp);
kernel/kprobes.c +1 -77
99 lines not on this page · raw message
kernel/test_kprobes.c +0 -94
106 lines not on this page · raw message
lib/Kconfig.debug +1 -1
@@ -1698,7 +1698,7 @@ config KPROBES_SANITY_TEST default n help This option provides for testing basic kprobes functionality on - boot. A sample kprobe, jprobe and kretprobe are inserted and + boot. Samples of kprobe and kretprobe are inserted and verified for functionality. Say N if you are unsure.
Masami Hiramatsu
maintainerPATCH v4 3/27 kprobes/x86: Remove jprobe implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/x86.
--- arch/x86/include/asm/kprobes.h | 3 - arch/x86/kernel/kprobes/core.c | 96 +--------------------------------------- 2 files changed, 3 insertions(+), 96 deletions(-)
arch/x86/include/asm/kprobes.h +0 -3
@@ -111,9 +111,6 @@ struct kprobe_ctlblk {unsigned long kprobe_status;unsigned long kprobe_old_flags;unsigned long kprobe_saved_flags; -unsigned long *jprobe_saved_sp; -struct pt_regs jprobe_saved_regs; -kprobe_opcode_t jprobes_stack[MAX_STACK_SIZE ];struct prev_kprobe prev_kprobe; };
arch/x86/kernel/kprobes/core.c +3 -93
114 lines not on this page · raw message
Masami Hiramatsu
maintainerPATCH v4 4/27 ARC: kprobes: Remove jprobe implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/arc.
--- arch/arc/include/asm/kprobes.h | 2 -- arch/arc/kernel/kprobes.c | 38 ++------------------------------------ 2 files changed, 2 insertions(+), 38 deletions(-)
arch/arc/include/asm/kprobes.h +0 -2
@@ -45,8 +45,6 @@ struct prev_kprobe {struct kprobe_ctlblk {unsigned int kprobe_status; -struct pt_regs jprobe_saved_regs; -char jprobes_stack[MAX_STACK_SIZE ];struct prev_kprobe prev_kprobe; };
arch/arc/kernel/kprobes.c +2 -36
@@ -225,10 +225,8 @@ int __kprobes arc_kprobe_handler(unsigned long addr, struct pt_regs *regs)/* If we have no pre-handler or it returned 0, we continue with * normal processing. If we have a pre-handler and it returned -* non-zero - which is expected from setjmp_pre_handler for -* jprobe, we return without single stepping and leave that to -* the break-handler which is invoked by a kprobe from -* jprobe_return +* non-zero - which means user handler setup registers to exit +* to another instruction, we must skip the single stepping. */ if (!p->pre_handler || !p->pre_handler (p, regs)) {setup_singlestep (p, regs);
@@ -386,38 +384,6 @@ int __kprobes kprobe_exceptions_notify(struct notifier_block *self,return ret; } -int __kprobes setjmp_pre_handler (struct kprobe *p,struct pt_regs *regs) -{ - struct jprobe *jp = container_of(p, struct jprobe, kp); -struct kprobe_ctlblk *kcb =get_kprobe_ctlblk (); -unsigned long sp_addr = regs->sp; - - kcb->jprobe_saved_regs = *regs; -memcpy (kcb->jprobes_stack, (void *)sp_addr,MIN_STACK_SIZE (sp_addr)); - regs->ret = (unsigned long )(jp->entry); - -return 1 ; -} - -void __kprobes jprobe_return (void ) -{ - __asm__ __volatile__("unimp_s" ); -return ; -} - -int __kprobes longjmp_break_handler (struct kprobe *p,struct pt_regs *regs) -{ -struct kprobe_ctlblk *kcb =get_kprobe_ctlblk (); -unsigned long sp_addr; - - *regs = kcb->jprobe_saved_regs; - sp_addr = regs->sp; -memcpy ((void *)sp_addr, kcb->jprobes_stack,MIN_STACK_SIZE (sp_addr)); -preempt_enable_no_resched (); - -return 1 ; -} -static void __used kretprobe_trampoline_holder (void ) { __asm__ __volatile__(".global kretprobe_trampoline\n"
Masami Hiramatsu
maintainerPATCH v4 5/27 ARM: kprobes: Remove jprobe arm implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/arm.
--- arch/arm/include/asm/kprobes.h | 2 - arch/arm/include/asm/probes.h | 1 arch/arm/probes/kprobes/core.c | 114 ---------------------------------------- 3 files changed, 117 deletions(-)
arch/arm/include/asm/kprobes.h +0 -2
@@ -44,8 +44,6 @@ struct prev_kprobe {struct kprobe_ctlblk {unsigned int kprobe_status;struct prev_kprobe prev_kprobe; -struct pt_regs jprobe_saved_regs; -char jprobes_stack[MAX_STACK_SIZE ]; };void arch_remove_kprobe (struct kprobe *);
arch/arm/include/asm/probes.h +0 -1
@@ -51,7 +51,6 @@ struct arch_probes_insn {* We assume one instruction can consume at most 64 bytes stack, which is * 'push {r0-r15}'. Instructions consume more or unknown stack space like * 'str r0, [sp, #-80]' and 'str r0, [sp, r1]' should be prohibit to probe. -* Both kprobe and jprobe use this macro. */ #define MAX_STACK_SIZE 64
arch/arm/probes/kprobes/core.c +0 -114
126 lines not on this page · raw message
Masami Hiramatsu
maintainerPATCH v4 6/27 arm64: kprobes: Remove jprobe implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/arm64.
--- arch/arm64/include/asm/kprobes.h | 1 - arch/arm64/kernel/probes/kprobes.c | 68 ------------------------------------ 2 files changed, 69 deletions(-)
arch/arm64/include/asm/kprobes.h +0 -1
@@ -48,7 +48,6 @@ struct kprobe_ctlblk {unsigned long saved_irqflag;struct prev_kprobe prev_kprobe;struct kprobe_step_ctx ss_ctx; -struct pt_regs jprobe_saved_regs; };void arch_remove_kprobe (struct kprobe *);
arch/arm64/kernel/probes/kprobes.c +0 -68
74 lines not on this page · raw message
Masami Hiramatsu
maintainerPATCH v4 7/27 powerpc/kprobes: Remove jprobe powerpc implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/powerpc. This also reverts commits
related __is_active_jprobe() function.
--- arch/powerpc/include/asm/kprobes.h | 2 - arch/powerpc/kernel/kprobes-ftrace.c | 15 ------- arch/powerpc/kernel/kprobes.c | 54 ------------------------ arch/powerpc/kernel/trace/ftrace_64_mprofile.S | 39 ++--------------- 4 files changed, 5 insertions(+), 105 deletions(-)
arch/powerpc/include/asm/kprobes.h +0 -2
@@ -88,7 +88,6 @@ struct prev_kprobe {struct kprobe_ctlblk {unsigned long kprobe_status;unsigned long kprobe_saved_msr; -struct pt_regs jprobe_saved_regs;struct prev_kprobe prev_kprobe; };
@@ -104,7 +103,6 @@ extern int kprobe_fault_handler(struct pt_regs *regs, int trapnr);extern int kprobe_handler (struct pt_regs *regs);extern int kprobe_post_handler (struct pt_regs *regs);#ifdef CONFIG_KPROBES_ON_FTRACE -extern int __is_active_jprobe (unsigned long addr);extern int skip_singlestep (struct kprobe *p,struct pt_regs *regs,struct kprobe_ctlblk *kcb);#else
arch/powerpc/kernel/kprobes-ftrace.c +0 -15
@@ -25,21 +25,6 @@#include <linux/preempt.h> #include <linux/ftrace.h> -/* -* This is called from ftrace code after invoking registered handlers to -* disambiguate regs->nip changes done by jprobes and livepatch. We check if -* there is an active jprobe at the provided address (mcount location). -*/ -int __is_active_jprobe (unsigned long addr) -{ -if (!preemptible ()) { -struct kprobe *p =raw_cpu_read (current_kprobe); -return (p && (unsigned long )p->addr == addr) ?1 :0 ; - } - -return 0 ; -} -static nokprobe_inline int __skip_singlestep(struct kprobe *p, struct pt_regs *regs,struct kprobe_ctlblk *kcb,unsigned long orig_nip)
arch/powerpc/kernel/kprobes.c +0 -54
60 lines not on this page · raw message
arch/powerpc/kernel/trace/ftrace_64_mprofile.S +5 -34
@@ -99,39 +99,13 @@ ftrace_call: bl ftrace_stub nop - /* Load the possibly modified NIP */ - ld r15, _NIP(r1) - + /* Load ctr with the possibly modified NIP */ + ld r3, _NIP(r1) + mtctr r3 #ifdef CONFIG_LIVEPATCH - cmpd r14, r15 /* has NIP been altered? */ + cmpd r14, r3 /* has NIP been altered? */ #endif -#if defined(CONFIG_LIVEPATCH) && defined(CONFIG_KPROBES_ON_FTRACE) - /* NIP has not been altered, skip over further checks */ - beq 1f - - /* Check if there is an active jprobe on us */ - subi r3, r14, 4 - bl __is_active_jprobe - nop - - /* - * If r3 == 1, then this is a kprobe/jprobe. - * else, this is livepatched function. - * - * The conditional branch for livepatch_handler below will use the - * result of this comparison. For kprobe/jprobe, we just need to branch to - * the new NIP, not call livepatch_handler. The branch below is bne, so we - * want CR0[EQ] to be true if this is a kprobe/jprobe. Which means we want - * CR0[EQ] = (r3 == 1). - */ - cmpdi r3, 1 -1: -#endif - - /* Load CTR with the possibly modified NIP */ - mtctr r15 - /* Restore gprs */ REST_GPR(0,r1) REST_10GPRS(2,r1)
@@ -149,10 +123,7 @@ ftrace_call: addi r1, r1, SWITCH_FRAME_SIZE #ifdef CONFIG_LIVEPATCH - /* - * Based on the cmpd or cmpdi above, if the NIP was altered and we're - * not on a kprobe/jprobe, then handle livepatch. - */ + /* Based on the cmpd above, if the NIP was altered handle livepatch */ bne- livepatch_handler #endif
Masami Hiramatsu
maintainerPATCH v4 8/27 ia64: kprobes: Remove jprobe implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/ia64.
Note that since ia64 jprobes code a bit different
from other arch, this keeps __IA64_BREAK_JPROBE
for checking break_handler. It will be removed
with break_handler() call afterwards.
--- arch/ia64/include/asm/kprobes.h | 2 - arch/ia64/kernel/Makefile | 2 - arch/ia64/kernel/jprobes.S | 90 --------------------------------------- arch/ia64/kernel/kprobes.c | 70 ------------------------------ 4 files changed, 1 insertion(+), 163 deletions(-) delete mode 100644 arch/ia64/kernel/jprobes.S
arch/ia64/include/asm/kprobes.h +0 -2
@@ -82,8 +82,6 @@ struct prev_kprobe {#define ARCH_PREV_KPROBE_SZ 2struct kprobe_ctlblk {unsigned long kprobe_status; -struct pt_regs jprobe_saved_regs; -unsigned long jprobes_saved_stacked_regs[MAX_PARAM_RSE_SIZE ];unsigned long *bsp;unsigned long cfm;atomic_t prev_kprobe_index;
arch/ia64/kernel/Makefile +1 -1
@@ -25,7 +25,7 @@ obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_PERFMON) += perfmon_default_smpl.o obj-$(CONFIG_IA64_CYCLONE) += cyclone.o obj-$(CONFIG_IA64_MCA_RECOVERY) += mca_recovery.o -obj-$(CONFIG_KPROBES) += kprobes.o jprobes.o +obj-$(CONFIG_KPROBES) += kprobes.o obj-$(CONFIG_DYNAMIC_FTRACE) += ftrace.o obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o crash.o obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
arch/ia64/kernel/jprobes.S deleted +0 -90
90 lines not on this page · raw message
arch/ia64/kernel/kprobes.c +0 -70
82 lines not on this page · raw message
Masami Hiramatsu
maintainerPATCH v4 9/27 MIPS: kprobes: Remove jprobe implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/mips.
--- arch/mips/include/asm/kprobes.h | 13 ----------- arch/mips/kernel/kprobes.c | 45 --------------------------------------- 2 files changed, 58 deletions(-)
arch/mips/include/asm/kprobes.h +0 -13
@@ -68,16 +68,6 @@ struct prev_kprobe {unsigned long saved_epc; }; -#define MAX_JPROBES_STACK_SIZE 128 -#define MAX_JPROBES_STACK_ADDR \ - (((unsigned long)current_thread_info()) + THREAD_SIZE - 32 - sizeof(struct pt_regs)) - -#define MIN_JPROBES_STACK_SIZE (ADDR ) \ - ((((ADDR) + MAX_JPROBES_STACK_SIZE) > MAX_JPROBES_STACK_ADDR) \ - ? MAX_JPROBES_STACK_ADDR - (ADDR) \ - : MAX_JPROBES_STACK_SIZE) - -#define SKIP_DELAYSLOT 0x0001/* per-cpu kprobe control block */
@@ -86,12 +76,9 @@ struct kprobe_ctlblk {unsigned long kprobe_old_SR;unsigned long kprobe_saved_SR;unsigned long kprobe_saved_epc; -unsigned long jprobe_saved_sp; -struct pt_regs jprobe_saved_regs;/* Per-thread fields, used while emulating branches */ unsigned long flags;unsigned long target_epc; -u8 jprobes_stack[MAX_JPROBES_STACK_SIZE ];struct prev_kprobe prev_kprobe; };
arch/mips/kernel/kprobes.c +0 -45
51 lines not on this page · raw message
Masami Hiramatsu
maintainerPATCH v4 10/27 s390/kprobes: Remove jprobe implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/s390.
--- arch/s390/include/asm/kprobes.h | 2 - arch/s390/kernel/kprobes.c | 59 +-------------------------------------- 2 files changed, 2 insertions(+), 59 deletions(-)
arch/s390/include/asm/kprobes.h +0 -2
@@ -68,8 +68,6 @@ struct kprobe_ctlblk {unsigned long kprobe_saved_imask;unsigned long kprobe_saved_ctl[3 ];struct prev_kprobe prev_kprobe; -struct pt_regs jprobe_saved_regs; -kprobe_opcode_t jprobes_stack[MAX_STACK_SIZE ]; };void arch_remove_kprobe (struct kprobe *p);
arch/s390/kernel/kprobes.c +2 -57
71 lines not on this page · raw message
Masami Hiramatsu
maintainerPATCH v4 11/27 sh: kprobes: Remove jprobe implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/sh.
--- arch/sh/include/asm/kprobes.h | 4 --- arch/sh/kernel/kprobes.c | 51 ----------------------------------------- 2 files changed, 55 deletions(-)
arch/sh/include/asm/kprobes.h +0 -4
@@ -27,7 +27,6 @@ struct kprobe;void arch_remove_kprobe (struct kprobe *);void kretprobe_trampoline (void ); -void jprobe_return_end (void );/* Architecture specific copy of original instruction*/ struct arch_specific_insn {
@@ -43,9 +42,6 @@ struct prev_kprobe {/* per-cpu kprobe control block */ struct kprobe_ctlblk {unsigned long kprobe_status; -unsigned long jprobe_saved_r15; -struct pt_regs jprobe_saved_regs; -kprobe_opcode_t jprobes_stack[MAX_STACK_SIZE ];struct prev_kprobe prev_kprobe; };
arch/sh/kernel/kprobes.c +0 -51
57 lines not on this page · raw message
Masami Hiramatsu
maintainerPATCH v4 12/27 sparc64: kprobes: Remove jprobe implementation
In reply to Masami HiramatsuRemove arch dependent setjump/longjump functions
and unused fields in kprobe_ctlblk for jprobes
from arch/sparc.
--- arch/sparc/include/asm/kprobes.h | 1 - arch/sparc/kernel/kprobes.c | 47 -------------------------------------- 2 files changed, 48 deletions(-)
arch/sparc/include/asm/kprobes.h +0 -1
@@ -44,7 +44,6 @@ struct kprobe_ctlblk {unsigned long kprobe_status;unsigned long kprobe_orig_tnpc;unsigned long kprobe_orig_tstate_pil; -struct pt_regs jprobe_saved_regs;struct prev_kprobe prev_kprobe; };
arch/sparc/kernel/kprobes.c +0 -47
53 lines not on this page · raw message
Masami Hiramatsu
maintainerPATCH v4 13/27 kprobes: Don't check the ->break_handler() in generic kprobes code
In reply to Masami HiramatsuDon't check the ->break_handler() from the core kprobes code, because it was only used by jprobes which got removed.
( In followup patches we'll remove the remaining calls in low level
arch handlers as well and remove the callback altogether. )
--- Documentation/kprobes.txt | 2 +- kernel/kprobes.c | 39 +++++---------------------------------- 2 files changed, 6 insertions(+), 35 deletions(-)
Documentation/kprobes.txt +1 -1
@@ -262,7 +262,7 @@ is optimized, that modification is ignored. Thus, if you want to tweak the kernel's execution path, you need to suppress optimization, using one of the following techniques: -- Specify an empty function for the kprobe's post_handler or break_handler. +- Specify an empty function for the kprobe's post_handler. or
kernel/kprobes.c +5 -34
@@ -627,8 +627,8 @@ static void optimize_kprobe(struct kprobe *p) (kprobe_disabled(p) || kprobes_all_disarmed)) return; -/* Both of break_handler and post_handler are not supported. */ -if (p->break_handler || p->post_handler) +/* kprobes with post_handler can not be optimized */ +if (p->post_handler)return ; op = container_of(p, struct optimized_kprobe, kp);
@@ -1116,20 +1116,6 @@ static int aggr_fault_handler(struct kprobe *p, struct pt_regs *regs, }NOKPROBE_SYMBOL (aggr_fault_handler); -static int aggr_break_handler (struct kprobe *p,struct pt_regs *regs) -{ -struct kprobe *cur =__this_cpu_read (kprobe_instance); -int ret =0 ; - -if (cur && cur->break_handler) { -if (cur->break_handler (cur, regs)) - ret =1 ; - } -reset_kprobe_instance (); -return ret; -} -NOKPROBE_SYMBOL (aggr_break_handler); -/* Walks the list and increments nmissed count for multiprobe case */ void kprobes_inc_nmissed_count (struct kprobe *p) {
@@ -1270,24 +1256,15 @@ static void cleanup_rp_inst(struct kretprobe *rp) }NOKPROBE_SYMBOL (cleanup_rp_inst); -/* -* Add the new probe to ap->list. Fail if this is the -* second break_handler at the address -*/ +/* Add the new probe to ap->list */ static int add_new_kprobe (struct kprobe *ap,struct kprobe *p) {BUG_ON (kprobe_gone (ap) ||kprobe_gone (p)); -if (p->break_handler || p->post_handler) +if (p->post_handler)unoptimize_kprobe (ap, true);/* Fall back to normal kprobe */ -if (p->break_handler) { -if (ap->break_handler) -return -EEXIST ; -list_add_tail_rcu (&p->list, &ap->list); - ap->break_handler = aggr_break_handler; - }else -list_add_rcu (&p->list, &ap->list); +list_add_rcu (&p->list, &ap->list);if (p->post_handler && !ap->post_handler) ap->post_handler = aggr_post_handler;
@@ -1310,8 +1287,6 @@ static void init_aggr_kprobe(struct kprobe *ap, struct kprobe *p)/* We don't care the kprobe which has gone. */ if (p->post_handler && !kprobe_gone (p)) ap->post_handler = aggr_post_handler; -if (p->break_handler && !kprobe_gone (p)) - ap->break_handler = aggr_break_handler;INIT_LIST_HEAD (&ap->list);INIT_HLIST_NODE (&ap->hlist);
@@ -1706,8 +1681,6 @@ static int __unregister_kprobe_top(struct kprobe *p) goto disarmed; else {/* If disabling probe has special handlers, update aggrprobe */ -if (p->break_handler && !kprobe_gone (p)) - ap->break_handler =NULL ;if (p->post_handler && !kprobe_gone (p)) {list_for_each_entry_rcu (list_p, &ap->list, list) {if ((list_p != p) && (list_p->post_handler))
@@ -1911,7 +1884,6 @@ int register_kretprobe(struct kretprobe *rp) rp->kp.pre_handler = pre_handler_kretprobe; rp->kp.post_handler =NULL ; rp->kp.fault_handler =NULL ; - rp->kp.break_handler =NULL ;/* Pre-allocate memory for max kretprobe instances */ if (rp->maxactive <=0 ) {
@@ -2034,7 +2006,6 @@ static void kill_kprobe(struct kprobe *p)list_for_each_entry_rcu (kp, &p->list, list) kp->flags |=KPROBE_FLAG_GONE ; p->post_handler =NULL ; - p->break_handler =NULL ;kill_optimized_kprobe (p); }/*
Masami Hiramatsu
maintainerPATCH v4 14/27 kprobes/x86: Don't call ->break_handler() in x86 kprobes
In reply to Masami HiramatsuDon't call ->break_handler() and remove break_handler related code since that was only used by jprobe which got removed.
--- arch/x86/include/asm/kprobes.h | 2 +- arch/x86/kernel/kprobes/common.h | 10 ---------- arch/x86/kernel/kprobes/core.c | 7 ------- arch/x86/kernel/kprobes/ftrace.c | 16 ++-------------- 4 files changed, 3 insertions(+), 32 deletions(-)
arch/x86/include/asm/kprobes.h +1 -1
@@ -78,7 +78,7 @@ struct arch_specific_insn {* boostable = true: This instruction has been boosted: we have * added a relative jump after the instruction copy in insn, * so no single-step and fixup are needed (unless there's -* a post_handler or break_handler). +* a post_handler). */ bool boostable;bool if_modifier;
arch/x86/kernel/kprobes/common.h +0 -10
@@ -105,14 +105,4 @@ static inline unsigned long __recover_optprobed_insn(kprobe_opcode_t *buf, unsig }#endif -#ifdef CONFIG_KPROBES_ON_FTRACE -extern int skip_singlestep (struct kprobe *p,struct pt_regs *regs, -struct kprobe_ctlblk *kcb); -#else -static inline int skip_singlestep (struct kprobe *p,struct pt_regs *regs, -struct kprobe_ctlblk *kcb) -{ -return 0 ; -} -#endif #endif
arch/x86/kernel/kprobes/core.c +0 -7
@@ -709,13 +709,6 @@ int kprobe_int3_handler(struct pt_regs *regs) regs->ip = (unsigned long )addr;preempt_enable_no_resched ();return 1 ; - }else if (kprobe_running ()) { - p =__this_cpu_read (current_kprobe); -if (p->break_handler && p->break_handler (p, regs)) { -if (!skip_singlestep (p, regs, kcb)) -setup_singlestep (p, regs, kcb,0 ); -return 1 ; - } }/* else: not a kprobe fault; let the kernel handle it */ preempt_enable_no_resched ();
arch/x86/kernel/kprobes/ftrace.c +2 -14
@@ -26,7 +26,7 @@#include "common.h" static nokprobe_inline -void __skip_singlestep(struct kprobe *p, struct pt_regs *regs, +void skip_singlestep(struct kprobe *p, struct pt_regs *regs,struct kprobe_ctlblk *kcb,unsigned long orig_ip) {/*
@@ -43,18 +43,6 @@ void __skip_singlestep(struct kprobe *p, struct pt_regs *regs, regs->ip = orig_ip; } -int skip_singlestep (struct kprobe *p,struct pt_regs *regs, -struct kprobe_ctlblk *kcb) -{ -if (kprobe_ftrace (p)) { -__skip_singlestep (p, regs, kcb,0 ); -preempt_enable_no_resched (); -return 1 ; - } -return 0 ; -} -NOKPROBE_SYMBOL (skip_singlestep); -/* Ftrace callback handler for kprobes -- called under preepmt disabed */ void kprobe_ftrace_handler (unsigned long ip,unsigned long parent_ip,struct ftrace_ops *ops,struct pt_regs *regs)
@@ -80,7 +68,7 @@ void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,__this_cpu_write (current_kprobe, p); kcb->kprobe_status =KPROBE_HIT_ACTIVE ;if (!p->pre_handler || !p->pre_handler (p, regs)) { -__skip_singlestep (p, regs, kcb, orig_ip); +skip_singlestep (p, regs, kcb, orig_ip);preempt_enable_no_resched (); }/*
Masami Hiramatsu
maintainerPATCH v4 15/27 ARC: kprobes: Don't call the ->break_handler() in ARC kprobes code
In reply to Masami HiramatsuDon't call the ->break_handler() from the ARC kprobes code, because it was only used by jprobes which got removed.
--- arch/arc/kernel/kprobes.c | 7 ------- 1 file changed, 7 deletions(-)
arch/arc/kernel/kprobes.c +0 -7
@@ -234,13 +234,6 @@ int __kprobes arc_kprobe_handler(unsigned long addr, struct pt_regs *regs) }return 1 ; - }else if (kprobe_running ()) { - p =__this_cpu_read (current_kprobe); -if (p->break_handler && p->break_handler (p, regs)) { -setup_singlestep (p, regs); - kcb->kprobe_status =KPROBE_HIT_SS ; -return 1 ; - } }/* no_kprobe: */
Masami Hiramatsu
maintainerPATCH v4 16/27 ARM: kprobes: Don't call the ->break_handler() in arm kprobes code
In reply to Masami HiramatsuDon't call the ->break_handler() from the arm kprobes code, because it was only used by jprobes which got removed.
--- arch/arm/probes/kprobes/core.c | 11 ----------- 1 file changed, 11 deletions(-)
arch/arm/probes/kprobes/core.c +0 -11
@@ -315,17 +315,6 @@ void __kprobes kprobe_handler(struct pt_regs *regs)reset_current_kprobe (); } } - }else if (cur) { -/* We probably hit a jprobe. Call its break handler. */ -if (cur->break_handler && cur->break_handler (cur, regs)) { - kcb->kprobe_status =KPROBE_HIT_SS ; -singlestep (cur, regs, kcb); -if (cur->post_handler) { - kcb->kprobe_status =KPROBE_HIT_SSDONE ; - cur->post_handler (cur, regs,0 ); - } - } -reset_current_kprobe (); }else {/* * The probe was removed and a race is in progress.
Masami Hiramatsu
maintainerPATCH v4 17/27 arm64: kprobes: Don't call the ->break_handler() in arm kprobes code
In reply to Masami HiramatsuDon't call the ->break_handler() from the arm kprobes code, because it was only used by jprobes which got removed.
--- arch/arm64/kernel/probes/kprobes.c | 8 -------- 1 file changed, 8 deletions(-)
arch/arm64/kernel/probes/kprobes.c +0 -8
@@ -408,14 +408,6 @@ static void __kprobes kprobe_handler(struct pt_regs *regs)return ; } } - }else if ((le32_to_cpu (*(kprobe_opcode_t *) addr) == -BRK64_OPCODE_KPROBES ) && cur_kprobe) { -/* We probably hit a jprobe. Call its break handler. */ -if (cur_kprobe->break_handler && - cur_kprobe->break_handler (cur_kprobe, regs)) { -setup_singlestep (cur_kprobe, regs, kcb,0 ); -return ; - } }/* * The breakpoint instruction was removed right
Masami Hiramatsu
maintainerPATCH v4 18/27 powerpc/kprobes: Don't call the ->break_handler() in arm kprobes code
In reply to Masami HiramatsuDon't call the ->break_handler() from the arm kprobes code, because it was only used by jprobes which got removed.
This also makes skip_singlestep() a static function since only ftrace-kprobe.c is using this function.
--- arch/powerpc/include/asm/kprobes.h | 10 ---------- arch/powerpc/kernel/kprobes-ftrace.c | 16 +++------------- arch/powerpc/kernel/kprobes.c | 31 +++++++++++-------------------- 3 files changed, 14 insertions(+), 43 deletions(-)
arch/powerpc/include/asm/kprobes.h +0 -10
@@ -102,16 +102,6 @@ extern int kprobe_exceptions_notify(struct notifier_block *self,extern int kprobe_fault_handler (struct pt_regs *regs,int trapnr);extern int kprobe_handler (struct pt_regs *regs);extern int kprobe_post_handler (struct pt_regs *regs); -#ifdef CONFIG_KPROBES_ON_FTRACE -extern int skip_singlestep (struct kprobe *p,struct pt_regs *regs, -struct kprobe_ctlblk *kcb); -#else -static inline int skip_singlestep (struct kprobe *p,struct pt_regs *regs, -struct kprobe_ctlblk *kcb) -{ -return 0 ; -} -#endif #else static inline int kprobe_handler (struct pt_regs *regs) {return 0 ; }static inline int kprobe_post_handler (struct pt_regs *regs) {return 0 ; }
arch/powerpc/kernel/kprobes-ftrace.c +3 -13
@@ -26,8 +26,8 @@#include <linux/ftrace.h> static nokprobe_inline -int __skip_singlestep(struct kprobe *p, struct pt_regs *regs, -struct kprobe_ctlblk *kcb,unsigned long orig_nip) +int skip_singlestep(struct kprobe *p, struct pt_regs *regs, +struct kprobe_ctlblk *kcb,unsigned long orig_nip) {/* * Emulate singlestep (and also recover regs->nip)
@@ -44,16 +44,6 @@ int __skip_singlestep(struct kprobe *p, struct pt_regs *regs,return 1 ; } -int skip_singlestep (struct kprobe *p,struct pt_regs *regs, -struct kprobe_ctlblk *kcb) -{ -if (kprobe_ftrace (p)) -return __skip_singlestep (p, regs, kcb,0 ); -else -return 0 ; -} -NOKPROBE_SYMBOL (skip_singlestep); -/* Ftrace callback handler for kprobes */ void kprobe_ftrace_handler (unsigned long nip,unsigned long parent_nip,struct ftrace_ops *ops,struct pt_regs *regs)
@@ -82,7 +72,7 @@ void kprobe_ftrace_handler(unsigned long nip, unsigned long parent_nip,__this_cpu_write (current_kprobe, p); kcb->kprobe_status =KPROBE_HIT_ACTIVE ;if (!p->pre_handler || !p->pre_handler (p, regs)) -__skip_singlestep (p, regs, kcb, orig_nip); +skip_singlestep (p, regs, kcb, orig_nip);else {/* * If pre_handler returns !0, it sets regs->nip and
arch/powerpc/kernel/kprobes.c +11 -20
@@ -317,25 +317,17 @@ int kprobe_handler(struct pt_regs *regs) }prepare_singlestep (p, regs);return 1 ; - }else { -if (*addr !=BREAKPOINT_INSTRUCTION ) { -/* If trap variant, then it belongs not to us */ -kprobe_opcode_t cur_insn = *addr; -if (is_trap (cur_insn)) - goto no_kprobe; -/* The breakpoint instruction was removed by -* another cpu right after we hit, no further -* handling of this interrupt is appropriate -*/ - ret =1 ; + }else if (*addr !=BREAKPOINT_INSTRUCTION ) { +/* If trap variant, then it belongs not to us */ +kprobe_opcode_t cur_insn = *addr; + +if (is_trap (cur_insn)) goto no_kprobe; - } - p =__this_cpu_read (current_kprobe); -if (p->break_handler && p->break_handler (p, regs)) { -if (!skip_singlestep (p, regs, kcb)) - goto ss_probe; - ret =1 ; - } +/* The breakpoint instruction was removed by +* another cpu right after we hit, no further +* handling of this interrupt is appropriate +*/ + ret =1 ; } goto no_kprobe; }
@@ -350,7 +342,7 @@ int kprobe_handler(struct pt_regs *regs)*/ kprobe_opcode_t cur_insn = *addr;if (is_trap (cur_insn)) - goto no_kprobe; + goto no_kprobe;/* * The breakpoint instruction was removed right * after we hit it. Another cpu has removed
@@ -370,7 +362,6 @@ int kprobe_handler(struct pt_regs *regs)/* handler has already set things up, so skip ss setup */ return 1 ; -ss_probe:if (p->ainsn.boostable >=0 ) { ret =try_to_emulate (p, regs);
Masami Hiramatsu
maintainerPATCH v4 19/27 ia64: kprobes: Don't call the ->break_handler() in ia64 kprobes code
In reply to Masami HiramatsuDon't call the ->break_handler() from the ia64 kprobes code, because it was only used by jprobes which got removed.
--- arch/ia64/include/uapi/asm/break.h | 1 - arch/ia64/kernel/kprobes.c | 10 ---------- 2 files changed, 11 deletions(-)
arch/ia64/include/uapi/asm/break.h +0 -1
@@ -14,7 +14,6 @@*/ #define __IA64_BREAK_KDB 0x80100#define __IA64_BREAK_KPROBE 0x81000/* .. 0x81fff */ -#define __IA64_BREAK_JPROBE 0x82000/* * OS-specific break numbers:
arch/ia64/kernel/kprobes.c +0 -10
@@ -817,14 +817,6 @@ static int __kprobes pre_kprobes_handler(struct die_args *args)prepare_ss (p, regs); kcb->kprobe_status =KPROBE_REENTER ;return 1 ; - }else if (args->err == __IA64_BREAK_JPROBE) { -/* -* jprobe instrumented function just completed -*/ - p =__this_cpu_read (current_kprobe); -if (p->break_handler && p->break_handler (p, regs)) { - goto ss_probe; - } }else if (!is_ia64_break_inst (regs)) {/* The breakpoint instruction was removed by * another cpu right after we hit, no further
@@ -867,7 +859,6 @@ static int __kprobes pre_kprobes_handler(struct die_args *args)*/ return 1 ; -ss_probe:#if !defined(CONFIG_PREEMPT )if (p->ainsn.inst_flag ==INST_FLAG_BOOSTABLE && !p->post_handler) {/* Boost up -- we can execute copied instructions directly */
@@ -990,7 +981,6 @@ int __kprobes kprobe_exceptions_notify(struct notifier_block *self,case DIE_BREAK :/* err is break number from ia64_bad_break() */ if ((args->err >>12 ) == (__IA64_BREAK_KPROBE >>12 ) - || args->err == __IA64_BREAK_JPROBE || args->err ==0 )if (pre_kprobes_handler (args)) ret =NOTIFY_STOP ;
Masami Hiramatsu
maintainerPATCH v4 20/27 MIPS: kprobes: Don't call the ->break_handler() in MIPS kprobes code
In reply to Masami HiramatsuDon't call the ->break_handler() from the MIPS kprobes code, because it was only used by jprobes which got removed.
--- arch/mips/kernel/kprobes.c | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-)
arch/mips/kernel/kprobes.c +7 -14
@@ -326,19 +326,13 @@ static int __kprobes kprobe_handler(struct pt_regs *regs)preempt_enable_no_resched (); }return 1 ; - }else { -if (addr->word != breakpoint_insn.word) { -/* -* The breakpoint instruction was removed by -* another cpu right after we hit, no further -* handling of this interrupt is appropriate -*/ - ret =1 ; - goto no_kprobe; - } - p =__this_cpu_read (current_kprobe); -if (p->break_handler && p->break_handler (p, regs)) - goto ss_probe; + }else if (addr->word != breakpoint_insn.word) { +/* +* The breakpoint instruction was removed by +* another cpu right after we hit, no further +* handling of this interrupt is appropriate +*/ + ret =1 ; } goto no_kprobe; }
@@ -367,7 +361,6 @@ static int __kprobes kprobe_handler(struct pt_regs *regs)return 1 ; } -ss_probe:prepare_singlestep (p, regs, kcb);if (kcb->flags &SKIP_DELAYSLOT ) { kcb->kprobe_status =KPROBE_HIT_SSDONE ;
Masami Hiramatsu
maintainerPATCH v4 21/27 s390/kprobes: Don't call the ->break_handler() in s390 kprobes code
In reply to Masami HiramatsuDon't call the ->break_handler() from the s390 kprobes code, because it was only used by jprobes which got removed.
--- arch/s390/kernel/kprobes.c | 20 -------------------- 1 file changed, 20 deletions(-)
arch/s390/kernel/kprobes.c +0 -20
@@ -332,26 +332,6 @@ static int kprobe_handler(struct pt_regs *regs) }enable_singlestep (kcb, regs, (unsigned long ) p->ainsn.insn);return 1 ; - }else if (kprobe_running ()) { - p =__this_cpu_read (current_kprobe); -if (p->break_handler && p->break_handler (p, regs)) { -/* -* Continuation after the jprobe completed and -* caused the jprobe_return trap. The jprobe -* break_handler "returns" to the original -* function that still has the kprobe breakpoint -* installed. We continue with single stepping. -*/ - kcb->kprobe_status =KPROBE_HIT_SS ; -enable_singlestep (kcb, regs, - (unsigned long ) p->ainsn.insn); -return 1 ; - }/* else: -* No kprobe at this address and the current kprobe -* has no break handler (no jprobe!). The kernel just -* exploded, let the standard trap handler pick up the -* pieces. -*/ }/* else: * No kprobe at this address and no active kprobe. The trap has * not been caused by a kprobe breakpoint. The race of breakpoint
Masami Hiramatsu
maintainerPATCH v4 22/27 sh: kprobes: Don't call the ->break_handler() in SH kprobes code
In reply to Masami HiramatsuDon't call the ->break_handler() from the SH kprobes code, because it was only used by jprobes which got removed.
--- arch/sh/kernel/kprobes.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-)
arch/sh/kernel/kprobes.c +1 -13
@@ -248,11 +248,6 @@ static int __kprobes kprobe_handler(struct pt_regs *regs)prepare_singlestep (p, regs); kcb->kprobe_status =KPROBE_REENTER ;return 1 ; - }else { - p =__this_cpu_read (current_kprobe); -if (p->break_handler && p->break_handler (p, regs)) { - goto ss_probe; - } } goto no_kprobe; }
@@ -281,7 +276,6 @@ static int __kprobes kprobe_handler(struct pt_regs *regs)/* handler has already set things up, so skip ss setup */ return 1 ; -ss_probe:prepare_singlestep (p, regs); kcb->kprobe_status =KPROBE_HIT_SS ;return 1 ;
@@ -508,14 +502,8 @@ int __kprobes kprobe_exceptions_notify(struct notifier_block *self,if (post_kprobe_handler (args->regs)) ret =NOTIFY_STOP ; }else { -if (kprobe_handler (args->regs)) { +if (kprobe_handler (args->regs)) ret =NOTIFY_STOP ; - }else { - p =__this_cpu_read (current_kprobe); -if (p->break_handler && - p->break_handler (p, args->regs)) - ret =NOTIFY_STOP ; - } } } }
Masami Hiramatsu
maintainerPATCH v4 23/27 sparc64: kprobes: Don't call the ->break_handler() in sparc64 kprobes code
In reply to Masami HiramatsuDon't call the ->break_handler() from the sparc64 kprobes code, because it was only used by jprobes which got removed.
--- arch/sparc/kernel/kprobes.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-)
arch/sparc/kernel/kprobes.c +2 -9
@@ -147,18 +147,12 @@ static int __kprobes kprobe_handler(struct pt_regs *regs) kcb->kprobe_status =KPROBE_REENTER ;prepare_singlestep (p, regs, kcb);return 1 ; - }else { -if (*(u32 *)addr !=BREAKPOINT_INSTRUCTION ) { + }else if (*(u32 *)addr !=BREAKPOINT_INSTRUCTION ) {/* The breakpoint instruction was removed by * another cpu right after we hit, no further * handling of this interrupt is appropriate */ - ret =1 ; - goto no_kprobe; - } - p =__this_cpu_read (current_kprobe); -if (p->break_handler && p->break_handler (p, regs)) - goto ss_probe; + ret =1 ; } goto no_kprobe; }
@@ -184,7 +178,6 @@ static int __kprobes kprobe_handler(struct pt_regs *regs)if (p->pre_handler && p->pre_handler (p, regs))return 1 ; -ss_probe:prepare_singlestep (p, regs, kcb); kcb->kprobe_status =KPROBE_HIT_SS ;return 1 ;
Masami Hiramatsu
maintainerPATCH v4 24/27 bpf: error-inject: kprobes: Clear current_kprobe and enable preempt in kprobe
In reply to Masami HiramatsuClear current_kprobe and enable preemption in kprobe even if pre_handler returns !0.
This simplifies function override using kprobes.
Jprobe used to require to keep the preemption disabled and keep current_kprobe until it returned to original function entry. For this reason kprobe_int3_handler() and similar arch dependent kprobe handers checks pre_handler result and exit without enabling preemption if the result is !0.
After removing the jprobe, Kprobes does not need to keep preempt disabled even if user handler returns !0 anymore.
But since the function override handler in error-inject and bpf is also returns !0 if it overrides a function, to balancing the preempt count, it enables preemption and reset current kprobe by itself.
That is a bad design that is very buggy. This fixes such unbalanced preempt-count and current_kprobes setting in kprobes, bpf and error-inject.
--- arch/arc/kernel/kprobes.c | 5 +++-- arch/arm/probes/kprobes/core.c | 10 +++++----- arch/arm64/kernel/probes/kprobes.c | 10 +++++----- arch/ia64/kernel/kprobes.c | 13 ++++--------- arch/mips/kernel/kprobes.c | 4 ++-- arch/powerpc/kernel/kprobes.c | 7 +++++-- arch/s390/kernel/kprobes.c | 7 ++++--- arch/sh/kernel/kprobes.c | 7 ++++--- arch/sparc/kernel/kprobes.c | 7 ++++--- arch/x86/kernel/kprobes/core.c | 4 ++++ arch/x86/kernel/kprobes/ftrace.c | 15 ++++++++------- kernel/fail_function.c | 3 --- kernel/trace/trace_kprobe.c | 11 +++-------- 13 files changed, 51 insertions(+), 52 deletions(-)
arch/arc/kernel/kprobes.c +3 -2
@@ -231,6 +231,9 @@ int __kprobes arc_kprobe_handler(unsigned long addr, struct pt_regs *regs)if (!p->pre_handler || !p->pre_handler (p, regs)) {setup_singlestep (p, regs); kcb->kprobe_status =KPROBE_HIT_SS ; + }else { +reset_current_kprobe (); +preempt_enable_no_resched (); }return 1 ;
@@ -442,9 +445,7 @@ static int __kprobes trampoline_probe_handler(struct kprobe *p,kretprobe_assert (ri, orig_ret_address, trampoline_address); regs->ret = orig_ret_address; -reset_current_kprobe ();kretprobe_hash_unlock (current, &flags); -preempt_enable_no_resched ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);
arch/arm/probes/kprobes/core.c +5 -5
@@ -300,10 +300,10 @@ void __kprobes kprobe_handler(struct pt_regs *regs)/* * If we have no pre-handler or it returned 0, we -* continue with normal processing. If we have a -* pre-handler and it returned non-zero, it prepped -* for calling the break_handler below on re-entry, -* so get out doing nothing more here. +* continue with normal processing. If we have a +* pre-handler and it returned non-zero, it will +* modify the execution path and no need to single +* stepping. Let's just reset current kprobe and exit. */ if (!p->pre_handler || !p->pre_handler (p, regs)) { kcb->kprobe_status =KPROBE_HIT_SS ;
@@ -312,8 +312,8 @@ void __kprobes kprobe_handler(struct pt_regs *regs) kcb->kprobe_status =KPROBE_HIT_SSDONE ; p->post_handler (p, regs,0 ); } -reset_current_kprobe (); } +reset_current_kprobe (); } }else {/*
arch/arm64/kernel/probes/kprobes.c +5 -5
@@ -395,9 +395,9 @@ static void __kprobes kprobe_handler(struct pt_regs *regs)/* * If we have no pre-handler or it returned 0, we * continue with normal processing. If we have a -* pre-handler and it returned non-zero, it prepped -* for calling the break_handler below on re-entry, -* so get out doing nothing more here. +* pre-handler and it returned non-zero, it will +* modify the execution path and no need to single +* stepping. Let's just reset current kprobe and exit. * * pre_handler can hit a breakpoint and can step thru * before return, keep PSTATE D-flag enabled until
@@ -405,8 +405,8 @@ static void __kprobes kprobe_handler(struct pt_regs *regs)*/ if (!p->pre_handler || !p->pre_handler (p, regs)) {setup_singlestep (p, regs, kcb,0 ); -return ; - } + }else +reset_current_kprobe (); } }/*
arch/ia64/kernel/kprobes.c +4 -9
@@ -478,12 +478,9 @@ int __kprobes trampoline_probe_handler(struct kprobe *p, struct pt_regs *regs)*/ break ; } -kretprobe_assert (ri, orig_ret_address, trampoline_address); -reset_current_kprobe ();kretprobe_hash_unlock (current, &flags); -preempt_enable_no_resched ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);
@@ -851,13 +848,11 @@ static int __kprobes pre_kprobes_handler(struct die_args *args)set_current_kprobe (p, kcb); kcb->kprobe_status =KPROBE_HIT_ACTIVE ; -if (p->pre_handler && p->pre_handler (p, regs)) -/* -* Our pre-handler is specifically requesting that we just -* do a return. This is used for both the jprobe pre-handler -* and the kretprobe trampoline -*/ +if (p->pre_handler && p->pre_handler (p, regs)) { +reset_current_kprobe (); +preempt_enable_no_resched ();return 1 ; + }#if !defined(CONFIG_PREEMPT )if (p->ainsn.inst_flag ==INST_FLAG_BOOSTABLE && !p->post_handler) {
arch/mips/kernel/kprobes.c +2 -2
@@ -358,6 +358,8 @@ static int __kprobes kprobe_handler(struct pt_regs *regs)if (p->pre_handler && p->pre_handler (p, regs)) {/* handler has already set things up, so skip ss setup */ +reset_current_kprobe (); +preempt_enable_no_resched ();return 1 ; }
@@ -543,9 +545,7 @@ static int __kprobes trampoline_probe_handler(struct kprobe *p,kretprobe_assert (ri, orig_ret_address, trampoline_address);instruction_pointer (regs) = orig_ret_address; -reset_current_kprobe ();kretprobe_hash_unlock (current, &flags); -preempt_enable_no_resched ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);
arch/powerpc/kernel/kprobes.c +5 -2
@@ -358,9 +358,12 @@ int kprobe_handler(struct pt_regs *regs) kcb->kprobe_status =KPROBE_HIT_ACTIVE ;set_current_kprobe (p, regs, kcb); -if (p->pre_handler && p->pre_handler (p, regs)) -/* handler has already set things up, so skip ss setup */ +if (p->pre_handler && p->pre_handler (p, regs)) { +/* handler changed execution path, so skip ss setup */ +reset_current_kprobe (); +preempt_enable_no_resched ();return 1 ; + }if (p->ainsn.boostable >=0 ) { ret =try_to_emulate (p, regs);
arch/s390/kernel/kprobes.c +4 -3
@@ -326,8 +326,11 @@ static int kprobe_handler(struct pt_regs *regs)*/ push_kprobe (kcb, p); kcb->kprobe_status =KPROBE_HIT_ACTIVE ; -if (p->pre_handler && p->pre_handler (p, regs)) +if (p->pre_handler && p->pre_handler (p, regs)) { +pop_kprobe (kcb); +preempt_enable_no_resched ();return 1 ; + } kcb->kprobe_status =KPROBE_HIT_SS ; }enable_singlestep (kcb, regs, (unsigned long ) p->ainsn.insn);
@@ -431,9 +434,7 @@ static int trampoline_probe_handler(struct kprobe *p, struct pt_regs *regs) regs->psw.addr = orig_ret_address; -pop_kprobe (get_kprobe_ctlblk ());kretprobe_hash_unlock (current, &flags); -preempt_enable_no_resched ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);
arch/sh/kernel/kprobes.c +4 -3
@@ -272,9 +272,12 @@ static int __kprobes kprobe_handler(struct pt_regs *regs)set_current_kprobe (p, regs, kcb); kcb->kprobe_status =KPROBE_HIT_ACTIVE ; -if (p->pre_handler && p->pre_handler (p, regs)) +if (p->pre_handler && p->pre_handler (p, regs)) {/* handler has already set things up, so skip ss setup */ +reset_current_kprobe (); +preempt_enable_no_resched ();return 1 ; + }prepare_singlestep (p, regs); kcb->kprobe_status =KPROBE_HIT_SS ;
@@ -352,8 +355,6 @@ int __kprobes trampoline_probe_handler(struct kprobe *p, struct pt_regs *regs) regs->pc = orig_ret_address;kretprobe_hash_unlock (current, &flags); -preempt_enable_no_resched (); -hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);kfree (ri);
arch/sparc/kernel/kprobes.c +4 -3
@@ -175,8 +175,11 @@ static int __kprobes kprobe_handler(struct pt_regs *regs)set_current_kprobe (p, regs, kcb); kcb->kprobe_status =KPROBE_HIT_ACTIVE ; -if (p->pre_handler && p->pre_handler (p, regs)) +if (p->pre_handler && p->pre_handler (p, regs)) { +reset_current_kprobe (); +preempt_enable_no_resched ();return 1 ; + }prepare_singlestep (p, regs, kcb); kcb->kprobe_status =KPROBE_HIT_SS ;
@@ -508,9 +511,7 @@ static int __kprobes trampoline_probe_handler(struct kprobe *p, regs->tpc = orig_ret_address; regs->tnpc = orig_ret_address +4 ; -reset_current_kprobe ();kretprobe_hash_unlock (current, &flags); -preempt_enable_no_resched ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);
arch/x86/kernel/kprobes/core.c +4 -0
@@ -694,6 +694,10 @@ int kprobe_int3_handler(struct pt_regs *regs)*/ if (!p->pre_handler || !p->pre_handler (p, regs))setup_singlestep (p, regs, kcb,0 ); +else { +reset_current_kprobe (); +preempt_enable_no_resched (); + }return 1 ; } }else if (*addr !=BREAKPOINT_INSTRUCTION ) {
arch/x86/kernel/kprobes/ftrace.c +8 -7
@@ -63,18 +63,19 @@ void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,/* Kprobe handler expects regs->ip = ip + 1 as breakpoint hit */ regs->ip = ip +sizeof (kprobe_opcode_t); -/* To emulate trap based kprobes, preempt_disable here */ -preempt_disable ();__this_cpu_write (current_kprobe, p); kcb->kprobe_status =KPROBE_HIT_ACTIVE ;if (!p->pre_handler || !p->pre_handler (p, regs)) {skip_singlestep (p, regs, kcb, orig_ip); -preempt_enable_no_resched (); + }else { +/* +* If pre_handler returns !0, this handler +* modifies regs->ip and goes back to there +* directly without single stepping. +* So let's just clear current kprobe. +*/ +__this_cpu_write (current_kprobe,NULL ); } -/* -* If pre_handler returns !0, it sets regs->ip and -* resets current kprobe, and keep preempt count +1. -*/ } }NOKPROBE_SYMBOL (kprobe_ftrace_handler);
kernel/fail_function.c +0 -3
@@ -184,9 +184,6 @@ static int fei_kprobe_handler(struct kprobe *kp, struct pt_regs *regs)if (should_fail (&fei_fault_attr,1 )) {regs_set_return_value (regs, attr->retval);override_function_with_return (regs); -/* Kprobe specific fixup */ -reset_current_kprobe (); -preempt_enable_no_resched ();return 1 ; }
kernel/trace/trace_kprobe.c +3 -8
@@ -1217,16 +1217,11 @@ kprobe_perf_func(struct trace_kprobe *tk, struct pt_regs *regs)/* * We need to check and see if we modified the pc of the -* pt_regs, and if so clear the kprobe and return 1 so that we -* don't do the single stepping. -* The ftrace kprobe handler leaves it up to us to re-enable -* preemption here before returning if we've modified the ip. +* pt_regs, and if so return 1 so that we don't do the +* single stepping. */ -if (orig_ip !=instruction_pointer (regs)) { -reset_current_kprobe (); -preempt_enable_no_resched (); +if (orig_ip !=instruction_pointer (regs))return 1 ; - }if (!ret)return 0 ; }
Re: [PATCH -tip v4 24/27] bpf: error-inject: kprobes: Clear current_kprobe and enable preempt in kprobe
Quoted 11 lines · Masami Hiramatsu
Clear
current_kprobeand enable preemption in kprobe even ifpre_handlerreturns !0.This simplifies function override using kprobes.
Jprobe used to require to keep the preemption disabled and keep
current_kprobeuntil it returned to original function entry. For this reasonkprobe_int3_handler()and similar arch dependent kprobe handers checkspre_handlerresult and exit without enabling preemption if the result is !0.
Quoted 3 lines
After removing the jprobe, Kprobes does not need to keep preempt disabled even if user handler returns !0 anymore.
I think the reason jprobes did it that way is to address architecture specific requirements when changing a function. So, without that infrastructure, I am not sure if we will be able to claim support for over-riding functions with kprobes. I am not sure if we want to claim that, but this is something we need to be clear on.
For powerpc, the current function override in error-inject works fine since the new function does nothing. But, if anyone wants to do more work in the replacement function, it won't work with the current approach.
Quoted 10 lines
But since the function override handler in error-inject and bpf is also returns !0 if it overrides a function, to balancing the preempt count, it enables preemption and reset current kprobe by itself.
That is a bad design that is very buggy. This fixes such unbalanced preempt-count and
current_kprobessetting in kprobes, bpf and error-inject.
Quoted 8 lines
Signed-off-by Masami Hiramatsu <[email protected]>--- arch/arc/kernel/kprobes.c | 5 +++-- arch/arm/probes/kprobes/core.c | 10 +++++----- arch/arm64/kernel/probes/kprobes.c | 10 +++++----- arch/ia64/kernel/kprobes.c | 13 ++++--------- arch/mips/kernel/kprobes.c | 4 ++-- arch/powerpc/kernel/kprobes.c | 7 +++++--
I think you should also update arch/powerpc/kernel/kprobes-ftrace.c
- Naveen
Masami Hiramatsu
maintainerRe: [PATCH -tip v4 24/27] bpf: error-inject: kprobes: Clear current_kprobe and enable preempt in kprobe
Quoted 16 lines · On Thu, 31 May 2018 16:25:38 +0530 "Naveen N. Rao"…
Quoted 11 lines · Masami Hiramatsu
Clear
current_kprobeand enable preemption in kprobe even ifpre_handlerreturns !0.This simplifies function override using kprobes.
Jprobe used to require to keep the preemption disabled and keep
current_kprobeuntil it returned to original function entry. For this reasonkprobe_int3_handler()and similar arch dependent kprobe handers checkspre_handlerresult and exit without enabling preemption if the result is !0.Quoted 3 lines
After removing the jprobe, Kprobes does not need to keep preempt disabled even if user handler returns !0 anymore.
Quoted 5 lines
I think the reason jprobes did it that way is to address architecture specific requirements when changing a function. So, without that infrastructure, I am not sure if we will be able to claim support for over-riding functions with kprobes. I am not sure if we want to claim that, but this is something we need to be clear on.
Really? as far as I can see, there seems no such architecture. The keeping preempt disabled is corresponding to keeping current_kprobe since the current_kprobe is per-cpu. This means if it is preempted before hitting break_handler and changed cpu core, we missed to handle current_kprobe and goes to panic. But if we don't need such "break back" (removing break_handler), we don't need to keep current_kprobe (because it is not handled afterwards).
Anyway, changing function execution path is a "one-way" change. We don't have a chance to fixup that disabled preemption and current_kprobe after returning to the new function. So current error-inject clears current_kprobe and enable preemption before returning !0 from its kprobe pre_handler.
This is just moving such needless operation from user-pre_handler to kprobes itself.
Quoted 4 lines
For powerpc, the current function override in error-inject works fine since the new function does nothing. But, if anyone wants to do more work in the replacement function, it won't work with the current approach.
If you are considering about TOC change etc. yes, it depends on the archtecture. As far as I know IA64 and powerpc will not allow to support changing execution path without special care. Other "flat and simple" function call architectures like x86, arm can change execution path without special care.
Anyway that is not related to this change. This is just a cleanup in total, something like re-balancing the operation.
Quoted 18 lines
Quoted 9 lines
But since the function override handler in error-inject and bpf is also returns !0 if it overrides a function, to balancing the preempt count, it enables preemption and reset current kprobe by itself.
That is a bad design that is very buggy. This fixes such unbalanced preempt-count and
current_kprobessetting in kprobes, bpf and error-inject.Quoted 8 lines
Signed-off-by Masami Hiramatsu <[email protected]>--- arch/arc/kernel/kprobes.c | 5 +++-- arch/arm/probes/kprobes/core.c | 10 +++++----- arch/arm64/kernel/probes/kprobes.c | 10 +++++----- arch/ia64/kernel/kprobes.c | 13 ++++--------- arch/mips/kernel/kprobes.c | 4 ++-- arch/powerpc/kernel/kprobes.c | 7 +++++--
Quoted 1 line
I think you should also update
arch/powerpc/kernel/kprobes-ftrace.c
Ah, good catch!! I'll fix that.
Thank you!
Signature · 1 line
Masami Hiramatsu <[email protected]>
Re: [PATCH -tip v4 24/27] bpf: error-inject: kprobes: Clear current_kprobe and enable preempt in kprobe
Quoted 19 lines · Masami Hiramatsu
Quoted 16 lines · On Thu, 31 May 2018 16:25:38 +0530 "Naveen N. Rao"…
Quoted 14 lines · Masami Hiramatsu
Clear
current_kprobeand enable preemption in kprobe even ifpre_handlerreturns !0.This simplifies function override using kprobes.
Jprobe used to require to keep the preemption disabled and keep
current_kprobeuntil it returned to original function entry. For this reasonkprobe_int3_handler()and similar arch dependent kprobe handers checkspre_handlerresult and exit without enabling preemption if the result is !0.After removing the jprobe, Kprobes does not need to keep preempt disabled even if user handler returns !0 anymore.
Quoted 5 lines
Quoted 1 line
I think the reason jprobes did it that way is to address architecture specific requirements when changing a function. So, without that infrastructure, I am not sure if we will be able to claim support for over-riding functions with kprobes. I am not sure if we want to claim that, but this is something we need to be clear on.
Really? as far as I can see, there seems no such architecture. The keeping preempt disabled is corresponding to keeping
current_kprobesince thecurrent_kprobeis per-cpu.
Right, and the reason for not resetting current_kprobe after kprobe handling is done is primarily for jprobes.
Quoted 5 lines
This means if it is preempted
before hittingbreak_handlerand changed cpu core, we missed to handlecurrent_kprobeand goes to panic. But if we don't need such "break back" (removingbreak_handler), we don't need to keepcurrent_kprobe(because it is not handled afterwards).
Agreed.
Quoted 2 lines
Anyway, changing function execution path is a "one-way" change.
This is the problem. With jprobes, over-riding a function was not a "one-way" change because it involves more than just changing the [n]ip. That is the reason we had setjmp/longjmp (aka break_handler).
Quoted 8 lines
We don't have a chance to fixup that disabled preemption and
current_kprobeafter returning to the new function. So current error-inject clearscurrent_kprobeand enable preemption before returning !0 from its kprobepre_handler.This is just moving such needless operation from user-pre_handler to kprobes itself.
Quoted 7 lines
Quoted 1 line
For powerpc, the current function override in error-inject works fine since the new function does nothing. But, if anyone wants to do more work in the replacement function, it won't work with the current approach.
If you are considering about TOC change etc. yes, it depends on the archtecture. As far as I know IA64 and powerpc will not allow to support changing execution path without special care. Other "flat and simple" function call architectures like x86, arm can change execution path without special care.
Yes, that's the concern. As I stated earlier, the only user seems to be error-injection where this is not a concern. I wanted this to be made clear.
I've since noticed that you are updating Documentation/kprobes.txt to make this clear in patch 24/27 in this series. So, I'm ok with the changes in this series.
Thanks,
Naveen
Masami Hiramatsu
maintainerRe: [PATCH -tip v4 24/27] bpf: error-inject: kprobes: Clear current_kprobe and enable preempt in kprobe
Quoted 67 lines · On Sat, 02 Jun 2018 17:28:05 +0530 "Naveen N. Rao"…
Quoted 25 lines · Masami Hiramatsu
Quoted 16 lines · On Thu, 31 May 2018 16:25:38 +0530 "Naveen N. Rao"…
Quoted 11 lines · Masami Hiramatsu
Clear
current_kprobeand enable preemption in kprobe even ifpre_handlerreturns !0.This simplifies function override using kprobes.
Jprobe used to require to keep the preemption disabled and keep
current_kprobeuntil it returned to original function entry. For this reasonkprobe_int3_handler()and similar arch dependent kprobe handers checkspre_handlerresult and exit without enabling preemption if the result is !0.Quoted 3 lines
After removing the jprobe, Kprobes does not need to keep preempt disabled even if user handler returns !0 anymore.
Quoted 5 lines
I think the reason jprobes did it that way is to address architecture specific requirements when changing a function. So, without that infrastructure, I am not sure if we will be able to claim support for over-riding functions with kprobes. I am not sure if we want to claim that, but this is something we need to be clear on.
Quoted 3 lines
Really? as far as I can see, there seems no such architecture. The keeping preempt disabled is corresponding to keeping
current_kprobesince thecurrent_kprobeis per-cpu.Right, and the reason for not resetting
current_kprobeafter kprobe handling is done is primarily for jprobes.Quoted 5 lines
This means if it is preempted
before hittingbreak_handlerand changed cpu core, we missed to handlecurrent_kprobeand goes to panic. But if we don't need such "break back" (removingbreak_handler), we don't need to keepcurrent_kprobe(because it is not handled afterwards).Agreed.
Quoted 2 lines
Anyway, changing function execution path is a "one-way" change.
This is the problem. With jprobes, over-riding a function was not a "one-way" change because it involves more than just changing the [n]ip. That is the reason we had setjmp/longjmp (aka
break_handler).Quoted 13 lines
We don't have a chance to fixup that disabled preemption and
current_kprobeafter returning to the new function. So current error-inject clearscurrent_kprobeand enable preemption before returning !0 from its kprobepre_handler.This is just moving such needless operation from user-pre_handler to kprobes itself.
Quoted 4 lines
For powerpc, the current function override in error-inject works fine since the new function does nothing. But, if anyone wants to do more work in the replacement function, it won't work with the current approach.
Quoted 5 lines
If you are considering about TOC change etc. yes, it depends on the archtecture. As far as I know IA64 and powerpc will not allow to support changing execution path without special care. Other "flat and simple" function call architectures like x86, arm can change execution path without special care.
Quoted 7 lines
Yes, that's the concern. As I stated earlier, the only user seems to be error-injection where this is not a concern. I wanted this to be made clear.
I've since noticed that you are updating
Documentation/kprobes.txtto make this clear in patch 24/27 in this series. So, I'm ok with the changes in this series.
I see your concern. Yeah, that's why I added [24/27] for clearly stating it.
Thanks,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Masami Hiramatsu
maintainerPATCH v4 25/27 x86: kprobes: Do not disable preempt on int3 path
In reply to Masami HiramatsuSince int3 and debug exception(for singlestep) are run with IRQ disabled and while running single stepping we drop IF from regs->flags, that path must not be preemptible. So we can remove the preempt disable/enable calls from that path.
---
Changes in v3:
- Split user-side changes to another patch
Changes in v2:
- Include user-side changes.
--- Documentation/kprobes.txt | 11 +++++------ arch/x86/kernel/kprobes/core.c | 18 ++++-------------- arch/x86/kernel/kprobes/opt.c | 1 - 3 files changed, 9 insertions(+), 21 deletions(-)
Documentation/kprobes.txt +5 -6
@@ -566,12 +566,11 @@ the same handler) may run concurrently on different CPUs. Kprobes does not use mutexes or allocate memory except during registration and unregistration. -Probe handlers are run with preemption disabled. Depending on the -architecture and optimization state, handlers may also run with -interrupts disabled (e.g., kretprobe handlers and optimized kprobe -handlers run without interrupt disabled on x86/x86-64). In any case, -your handler should not yield the CPU (e.g., by attempting to acquire -a semaphore). +Probe handlers are run with preemption disabled or interrupt disabled, +which depends on the architecture and optimization state. (e.g., +kretprobe handlers and optimized kprobe handlers run without interrupt +disabled on x86/x86-64). In any case, your handler should not yield +the CPU (e.g., by attempting to acquire a semaphore, or waiting I/O). Since a return probe is implemented by replacing the return address with the trampoline's address, stack backtraces and calls
arch/x86/kernel/kprobes/core.c +4 -14
@@ -594,7 +594,6 @@ static void setup_singlestep(struct kprobe *p, struct pt_regs *regs,* stepping. */ regs->ip = (unsigned long )p->ainsn.insn; -preempt_enable_no_resched (); return; }#endif
@@ -667,12 +666,10 @@ int kprobe_int3_handler(struct pt_regs *regs) addr = (kprobe_opcode_t *)(regs->ip -sizeof (kprobe_opcode_t));/* -* We don't want to be preempted for the entire -* duration of kprobe processing. We conditionally -* re-enable preemption at the end of this function, -* and also in reenter_kprobe() and setup_singlestep(). +* We don't want to be preempted for the entire duration of kprobe +* processing. Since int3 and debug trap disables irqs and we clear +* IF while singlestepping, it must be no preemptible. */ -preempt_disable (); kcb =get_kprobe_ctlblk (); p =get_kprobe (addr);
@@ -694,10 +691,8 @@ int kprobe_int3_handler(struct pt_regs *regs)*/ if (!p->pre_handler || !p->pre_handler (p, regs))setup_singlestep (p, regs, kcb,0 ); -else { +else reset_current_kprobe (); -preempt_enable_no_resched (); - }return 1 ; } }else if (*addr !=BREAKPOINT_INSTRUCTION ) {
@@ -711,11 +706,9 @@ int kprobe_int3_handler(struct pt_regs *regs)* the original instruction. */ regs->ip = (unsigned long )addr; -preempt_enable_no_resched ();return 1 ; }/* else: not a kprobe fault; let the kernel handle it */ -preempt_enable_no_resched ();return 0 ; }NOKPROBE_SYMBOL (kprobe_int3_handler);
@@ -966,8 +959,6 @@ int kprobe_debug_handler(struct pt_regs *regs) }reset_current_kprobe (); out: -preempt_enable_no_resched (); -/* * if somebody else is singlestepping across a probe point, flags * will have TF set, in which case, continue the remaining processing
@@ -1014,7 +1005,6 @@ int kprobe_fault_handler(struct pt_regs *regs, int trapnr)restore_previous_kprobe (kcb); elsereset_current_kprobe (); -preempt_enable_no_resched (); }else if (kcb->kprobe_status ==KPROBE_HIT_ACTIVE || kcb->kprobe_status ==KPROBE_HIT_SSDONE ) {/*
arch/x86/kernel/kprobes/opt.c +0 -1
@@ -491,7 +491,6 @@ int setup_detour_execution(struct kprobe *p, struct pt_regs *regs, int reenter) regs->ip = (unsigned long )op->optinsn.insn +TMPL_END_IDX ;if (!reenter)reset_current_kprobe (); -preempt_enable_no_resched ();return 1 ; }return 0 ;
Masami Hiramatsu
maintainerPATCH v4 26/27 Documentation: kprobes: Add how to change the execution path
In reply to Masami HiramatsuAdd a section that explaining how to change the execution path with kprobes.
--- Documentation/kprobes.txt | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
Documentation/kprobes.txt +20 -0
@@ -80,6 +80,26 @@ After the instruction is single-stepped, Kprobes executes the "post_handler," if any, that is associated with the kprobe. Execution then continues with the instruction following the probepoint. +Changing Execution Path +----------------------- + +Since the kprobes can probe into a running kernel code, it can change +the register set, including instruction pointer. This operation +requires maximum attention, such as keeping the stack frame, recovering +execution path etc. Since it is operated on running kernel and need deep +knowladge of the archtecture and concurrent computing, you can easily +shot your foot. + +If you change the instruction pointer (and set up other related +registers) in pre_handler, you must return !0 so that the kprobes +stops single stepping and just returns to given address. +This also means post_handler should not be called anymore. + +Note that this operation may be harder on some architectures which +use TOC (Table of Contents) for function call, since you have to +setup new TOC for your function in your module, and recover old +one after back from it. + Return Probes -------------
Masami Hiramatsu
maintainerPATCH v4 27/27 kprobes: Remove jprobe stub API
In reply to Masami HiramatsuRemove jprobe stub APIs from linux/kprobes.h since the jprobe implementation was completely gone.
--- include/linux/kprobes.h | 50 ----------------------------------------------- 1 file changed, 50 deletions(-)
include/linux/kprobes.h +0 -50
@@ -63,7 +63,6 @@ struct pt_regs;struct kretprobe ;struct kretprobe_instance ;typedef int (*kprobe_pre_handler_t ) (struct kprobe *,struct pt_regs *); -typedef int (*kprobe_break_handler_t ) (struct kprobe *,struct pt_regs *);typedef void (*kprobe_post_handler_t ) (struct kprobe *,struct pt_regs *,unsigned long flags); typedef int (*kprobe_fault_handler_t) (struct kprobe *, struct pt_regs *,
@@ -101,12 +100,6 @@ struct kprobe {*/ kprobe_fault_handler_t fault_handler; -/* -* ... called if breakpoint trap occurs in probe handler. -* Return 1 if it handled break, otherwise kernel will see it. -*/ -kprobe_break_handler_t break_handler; -/* Saved opcode (which has been replaced with breakpoint) */ kprobe_opcode_t opcode;
@@ -155,24 +148,6 @@ static inline int kprobe_ftrace(struct kprobe *p) }/* -* Special probe type that uses setjmp-longjmp type tricks to resume -* execution at a specified entry with a matching prototype corresponding -* to the probed function - a trick to enable arguments to become -* accessible seamlessly by probe handling logic. -* Note: -* Because of the way compilers allocate stack space for local variables -* etc upfront, regardless of sub-scopes within a function, this mirroring -* principle currently works only for probes placed on function entry points. -*/ -struct jprobe { -struct kprobe kp; -void *entry;/* probe handling code to jump to */ -}; - -/* For backward compatibility with old code using JPROBE_ENTRY() */ -#define JPROBE_ENTRY (handler) (handler) - -/* * Function-return probe - * Note: * User needs to provide a handler function, and initialize maxactive.
@@ -436,9 +411,6 @@ static inline void unregister_kprobe(struct kprobe *p)static inline void unregister_kprobes (struct kprobe **kps,int num) { } -static inline void jprobe_return (void ) -{ -}static inline int register_kretprobe (struct kretprobe *rp) {return -ENOSYS ;
@@ -465,20 +437,6 @@ static inline int enable_kprobe(struct kprobe *kp) return -ENOSYS; }#endif /* CONFIG_KPROBES */ -static inline int register_jprobe (struct jprobe *p) -{ -return -ENOSYS ; -} -static inline int register_jprobes (struct jprobe **jps,int num) -{ -return -ENOSYS ; -} -static inline void unregister_jprobe (struct jprobe *p) -{ -} -static inline void unregister_jprobes (struct jprobe **jps,int num) -{ -}static inline int disable_kretprobe (struct kretprobe *rp) {return disable_kprobe (&rp->kp);
@@ -487,14 +445,6 @@ static inline int enable_kretprobe(struct kretprobe *rp) {return enable_kprobe (&rp->kp); } -static inline int disable_jprobe (struct jprobe *jp) -{ -return -ENOSYS ; -} -static inline int enable_jprobe (struct jprobe *jp) -{ -return -ENOSYS ; -}#ifndef CONFIG_KPROBES static inline bool is_kprobe_insn_slot (unsigned long addr)
Masami Hiramatsu
maintainerHi Naveen,
Sorry, I might miss to add you on CC list. Could you test and review this on powerpc?
Thank you,
Quoted 151 lines · On Mon, 28 May 2018 15:59:09 +0900 Masami Hiramatsu <[email protected]> wrote:
Hello,
Since we decided to remove jprobe from kernel last year, its APIs are disabled and we worked on moving in-kernel jprobe users to kprobes or trace-events. And now no jprobe users are here anymore.
This is the 4th version of the series for removing jprobe. Previous version is here:
https://lkml.org/lkml/2018/5/16/1052
Unlike previous versions, this version removes jprobe from all architectures, as Ingo asked me in the previous thread.
I marked this as RFC again because it introduced changes for many arch. It should be reviewed by some arch which is not simply removing code (e.g. powerpc, arm).
BTW, this series are based on -tip tree as same as its previous version, and I'm sure it can be applied to linus tree/linux-next too.
Series structure
=====
Basically this series introduces 3 major changes;- Remove jprobe implementation ([2/27] - [12/27])
- Remove
break_handler()related code ([13/27] - [23/27]) This should be done after (or with) removing jprobe becausebreak_handleris only used by jprobe. (I'm not so sure, should I merge above patches into one arch-wide patch as next one does?)- Clean
current_kprobeand enable preempt ifpre_handler()returns !0 ([24/27] and [25/27]<- x86 specific update) This also depends on above patches because thosecurrent_kprobeand preemption are expected to be adjusted by jprobe implementation viabreak_handler.And some minor changes;
- Document cleanup and update ([1/27], [26/27])
And finally remove jprobe stub APIs and
break_handlerfromkprobes.h([27/27]). Of course this depends on above patches.Since removing jprobes and
break_handlerrelated code involve archtecture specific changes in some archs, I splitted it for each arch. But "cleancurrent_kprobeand enable preempt ifpre_handler()returns !0" patch modifies all architectures, since it changes expected kprobe handler behavior. I think this would be better done in one patch for consistency.Result
=====
I've tested it with kprobe sanity test on x86-64, and arm64 but for other archs, I just did cross-build test.With this series, we finally cleanup all jprobe code and
break_handleras below.$ git grep -wi break_handler | wc -l 0 $ git grep -wi jprobe | wc -l 0Actually, there is one place where mentioning jprobe,
Documentation/kprobes.txtexplains jprobe is deprecated and how to migrate jprobe user to ftrace or kprobe. I didn't remove it because it might be useful for some users.Thank you,
---
Masami Hiramatsu (27):
Documentation/kprobes: Fix to remove remaining jprobe
kprobes: Remove jprobe API implementation
kprobes/x86: Remove jprobe implementation
ARC: kprobes: Remove jprobe implementation
ARM: kprobes: Remove jprobe arm implementation
arm64: kprobes: Remove jprobe implementation
powerpc/kprobes: Remove jprobe powerpc implementation
ia64: kprobes: Remove jprobe implementation
MIPS: kprobes: Remove jprobe implementation
s390/kprobes: Remove jprobe implementation
sh: kprobes: Remove jprobe implementation
sparc64: kprobes: Remove jprobe implementation
kprobes: Don't check the ->break_handler() in generic kprobes code kprobes/x86: Don't call ->break_handler() in x86 kprobes
ARC: kprobes: Don't call the ->break_handler() in ARC kprobes code
ARM: kprobes: Don't call the ->break_handler() in arm kprobes code
arm64: kprobes: Don't call the ->break_handler() in arm kprobes code powerpc/kprobes: Don't call the ->break_handler() in arm kprobes code
ia64: kprobes: Don't call the ->break_handler() in ia64 kprobes code
MIPS: kprobes: Don't call the ->break_handler() in MIPS kprobes code s390/kprobes: Don't call the ->break_handler() in s390 kprobes code
sh: kprobes: Don't call the ->break_handler() in SH kprobes code
sparc64: kprobes: Don't call the ->break_handler() in sparc64 kprobes code
bpf: error-inject: kprobes: Clearcurrent_kprobeand enable preempt in kprobe
x86: kprobes: Do not disable preempt on int3 path
Documentation: kprobes: Add how to change the execution path
kprobes: Remove jprobe stub APIDocumentation/kprobes.txt | 35 +++++- arch/arc/include/asm/kprobes.h | 2 arch/arc/kernel/kprobes.c | 50 +-------- arch/arm/include/asm/kprobes.h | 2 arch/arm/include/asm/probes.h | 1 arch/arm/probes/kprobes/core.c | 135 +----------------------- arch/arm64/include/asm/kprobes.h | 1 arch/arm64/kernel/probes/kprobes.c | 86 +-------------- arch/ia64/include/asm/kprobes.h | 2 arch/ia64/include/uapi/asm/break.h | 1 arch/ia64/kernel/Makefile | 2 arch/ia64/kernel/jprobes.S | 90 ---------------- arch/ia64/kernel/kprobes.c | 93 +---------------- arch/mips/include/asm/kprobes.h | 13 -- arch/mips/kernel/kprobes.c | 70 ++---------- arch/powerpc/include/asm/kprobes.h | 12 -- arch/powerpc/kernel/kprobes-ftrace.c | 31 +----- arch/powerpc/kernel/kprobes.c | 92 +++------------- arch/powerpc/kernel/trace/ftrace_64_mprofile.S | 39 +------ arch/s390/include/asm/kprobes.h | 2 arch/s390/kernel/kprobes.c | 86 +-------------- arch/sh/include/asm/kprobes.h | 4 - arch/sh/kernel/kprobes.c | 72 +------------ arch/sparc/include/asm/kprobes.h | 1 arch/sparc/kernel/kprobes.c | 65 +----------- arch/x86/include/asm/kprobes.h | 5 - arch/x86/kernel/kprobes/common.h | 10 -- arch/x86/kernel/kprobes/core.c | 119 +-------------------- arch/x86/kernel/kprobes/ftrace.c | 31 ++---- arch/x86/kernel/kprobes/opt.c | 1 include/linux/kprobes.h | 53 --------- kernel/fail_function.c | 3 - kernel/kprobes.c | 115 +------------------- kernel/test_kprobes.c | 94 ----------------- kernel/trace/trace_kprobe.c | 11 +- lib/Kconfig.debug | 2 36 files changed, 125 insertions(+), 1306 deletions(-) delete mode 100644 arch/ia64/kernel/jprobes.S--
Masami Hiramatsu (Linaro)
Signature · 1 line
Masami Hiramatsu <[email protected]>
Hi Masami,
Quoted 3 lines · Masami Hiramatsu
Hi Naveen,
Sorry, I might miss to add you on CC list. Could you test and review this on powerpc?
Sure, thanks for letting me know. If possible, please mark powerpc patches to [email protected] so that it gets good review.
- Naveen