arm64: ftrace with regs
Subsystems (15)
cover arm64: ftrace with regs
Try to make the arm64 ftrace with regs merged.
The later two patches are from Torsten.
The whole series applies cleanly on v5.3-rc5
Jisheng Zhang (1):
ftrace: introdue ftrace_call_init
Torsten Duwe (2):
arm64: implement ftrace with regs
arm64: use -fpatchable-function-entry if available
arch/arm64/Kconfig | 2 + arch/arm64/Makefile | 5 ++ arch/arm64/include/asm/ftrace.h | 12 ++- arch/arm64/include/asm/module.h | 3 +- arch/arm64/kernel/entry-ftrace.S | 125 ++++++++++++++++++++++++++-- arch/arm64/kernel/ftrace.c | 138 +++++++++++++++++++++++-------- arch/arm64/kernel/module-plts.c | 3 +- arch/arm64/kernel/module.c | 2 +- include/linux/ftrace.h | 1 + kernel/module.c | 7 +- kernel/trace/ftrace.c | 4 + 11 files changed, 257 insertions(+), 45 deletions(-)
Signature · 1 line
2.23.0.rc1
PATCH 1/3 ftrace: introdue ftrace_call_init
On some arch, the FTRACE_WITH_REGS is implemented with gcc's
-fpatchable-function-entry (=2), gcc adds 2 NOPs at the beginning
of each function, so this makes the MCOUNT_ADDR useless. In ftrace common framework, MCOUNT_ADDR is mostly used to "init" the nop, so let's introcude ftrace_call_init().
--- include/linux/ftrace.h | 1 + kernel/trace/ftrace.c | 4 ++++ 2 files changed, 5 insertions(+)
include/linux/ftrace.h +1 -0
@@ -458,6 +458,7 @@ extern void ftrace_regs_caller(void);extern void ftrace_call (void );extern void ftrace_regs_call (void );extern void mcount_call (void ); +extern int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec);void ftrace_modify_all_code (int command);
kernel/trace/ftrace.c +4 -0
@@ -2500,7 +2500,11 @@ ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec)if (unlikely (ftrace_disabled))return 0 ; +#ifdef MCOUNT_ADDR ret =ftrace_make_nop (mod, rec,MCOUNT_ADDR ); +#else + ret =ftrace_call_init (mod, rec); +#endif if (ret) { ftrace_bug_type =FTRACE_BUG_INIT ;ftrace_bug (ret, rec);
Signature · 1 line
2.23.0.rc1
Re: [PATCH 1/3] ftrace: introdue ftrace_call_init
Quoted 29 lines · On Mon, 19 Aug 2019 19:16:22 +0800 Jisheng Zhang wrote:
On some arch, the
FTRACE_WITH_REGSis implemented with gcc's
-fpatchable-function-entry (=2), gcc adds 2 NOPs at the beginning
of each function, so this makes theMCOUNT_ADDRuseless. In ftrace common framework,MCOUNT_ADDRis mostly used to "init" the nop, so let's introcudeftrace_call_init().Signed-off-by Jisheng Zhang <[email protected]>--- include/linux/ftrace.h | 1 + kernel/trace/ftrace.c | 4 ++++ 2 files changed, 5 insertions(+)include/linux/ftrace.h +1
-0@@ -458,6 +458,7 @@ extern void ftrace_regs_caller(void);extern void ftrace_call (void );extern void ftrace_regs_call (void );extern void mcount_call (void ); +extern int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec);void ftrace_modify_all_code (int command);kernel/trace/ftrace.c +0
-0
kernel/trace/ftrace.c ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec) · line 2507
Quoted 8 lines
kernel/trace/ftrace.c +4
-0@@ -2500,7 +2500,11 @@ ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec)if (unlikely (ftrace_disabled))return 0 ; +#ifdef MCOUNT_ADDR ret =ftrace_make_nop (mod, rec,MCOUNT_ADDR ); +#else + ret =ftrace_call_init (mod, rec); +#endif
If we want to remove MCOUNT_ADDR from all arch, I could cook patches for this.
Re: [PATCH 1/3] ftrace: introdue ftrace_call_init
In reply to Jisheng ZhangHi,
Quoted 30 lines · Jisheng Zhang
On some arch, the
FTRACE_WITH_REGSis implemented with gcc's
-fpatchable-function-entry (=2), gcc adds 2 NOPs at the beginning
of each function, so this makes theMCOUNT_ADDRuseless. In ftrace common framework,MCOUNT_ADDRis mostly used to "init" the nop, so let's introcudeftrace_call_init().Signed-off-by Jisheng Zhang <[email protected]>--- include/linux/ftrace.h | 1 + kernel/trace/ftrace.c | 4 ++++ 2 files changed, 5 insertions(+)include/linux/ftrace.h +1
-0@@ -458,6 +458,7 @@ extern void ftrace_regs_caller(void);extern void ftrace_call (void );extern void ftrace_regs_call (void );extern void mcount_call (void ); +extern int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec);void ftrace_modify_all_code (int command);kernel/trace/ftrace.c +0
-0@@ -2500,7 +2500,11 @@ ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec)if (unlikely (ftrace_disabled))
kernel/trace/ftrace.c ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec) · line 2510
Quoted 10 lines
kernel/trace/ftrace.c +4
-0return 0 ; +#ifdef MCOUNT_ADDR ret =ftrace_make_nop (mod, rec,MCOUNT_ADDR ); +#else + ret =ftrace_call_init (mod, rec); +#endif if (ret) { ftrace_bug_type =FTRACE_BUG_INIT ;ftrace_bug (ret, rec);
I may be missing something, but the patch does not seem to be complete. There is no ftrace_call_init() implemented. MCOUNT_ADDR is still defined, so it does not change much. I don't think it is what Mark had in his mind.
Miroslav
Re: [PATCH 1/3] ftrace: introdue ftrace_call_init
Hi,
Quoted 38 lines · On Tue, 20 Aug 2019 11:27:38 +0200 (CEST) Miroslav Benes <[email protected]> wrote:
Hi,
Quoted 32 lines · Jisheng Zhang
On some arch, the
FTRACE_WITH_REGSis implemented with gcc's
-fpatchable-function-entry (=2), gcc adds 2 NOPs at the beginning
of each function, so this makes theMCOUNT_ADDRuseless. In ftrace common framework,MCOUNT_ADDRis mostly used to "init" the nop, so let's introcudeftrace_call_init().Signed-off-by Jisheng Zhang <[email protected]>--- include/linux/ftrace.h | 1 + kernel/trace/ftrace.c | 4 ++++ 2 files changed, 5 insertions(+)include/linux/ftrace.h +1
-0@@ -458,6 +458,7 @@ extern void ftrace_regs_caller(void);extern void ftrace_call (void );extern void ftrace_regs_call (void );extern void mcount_call (void ); +extern int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec);void ftrace_modify_all_code (int command);kernel/trace/ftrace.c +0
-0@@ -2500,7 +2500,11 @@ ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec)if (unlikely (ftrace_disabled))return 0 ;
Quoted 13 lines
kernel/trace/ftrace.c ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec) · line 2510
Quoted 8 lines
kernel/trace/ftrace.c +4
-0+#ifdef MCOUNT_ADDR ret =ftrace_make_nop (mod, rec,MCOUNT_ADDR ); +#else + ret =ftrace_call_init (mod, rec); +#endif if (ret) { ftrace_bug_type =FTRACE_BUG_INIT ;ftrace_bug (ret, rec);I may be missing something, but the patch does not seem to be complete. There is no
ftrace_call_init()implemented.MCOUNT_ADDRis still defined, so it does not change much. I don't think it is what Mark had in his mind.
Yes, except arm64, MCOUNT_ADDR is still defined in other arch. If we want to remove MCOUNT_ADDR from all arch, I will cook patches for this purpose.
Thanks
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/3] ftrace: introdue ftrace_call_init
In reply to Miroslav BenesHi all,
Quoted 37 lines · On Tue, 20 Aug 2019 11:27:38 +0200 (CEST) Miroslav Benes wrote:
Hi,
Quoted 32 lines · Jisheng Zhang
On some arch, the
FTRACE_WITH_REGSis implemented with gcc's
-fpatchable-function-entry (=2), gcc adds 2 NOPs at the beginning
of each function, so this makes theMCOUNT_ADDRuseless. In ftrace common framework,MCOUNT_ADDRis mostly used to "init" the nop, so let's introcudeftrace_call_init().Signed-off-by Jisheng Zhang <[email protected]>--- include/linux/ftrace.h | 1 + kernel/trace/ftrace.c | 4 ++++ 2 files changed, 5 insertions(+)include/linux/ftrace.h +1
-0@@ -458,6 +458,7 @@ extern void ftrace_regs_caller(void);extern void ftrace_call (void );extern void ftrace_regs_call (void );extern void mcount_call (void ); +extern int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec);void ftrace_modify_all_code (int command);kernel/trace/ftrace.c +0
-0@@ -2500,7 +2500,11 @@ ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec)if (unlikely (ftrace_disabled))return 0 ;
Quoted 13 lines
kernel/trace/ftrace.c ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec) · line 2510
Quoted 8 lines
kernel/trace/ftrace.c +4
-0+#ifdef MCOUNT_ADDR ret =ftrace_make_nop (mod, rec,MCOUNT_ADDR ); +#else + ret =ftrace_call_init (mod, rec); +#endif if (ret) { ftrace_bug_type =FTRACE_BUG_INIT ;ftrace_bug (ret, rec);I may be missing something, but the patch does not seem to be complete. There is no
ftrace_call_init()implemented.MCOUNT_ADDRis still defined, so it does not change much. I don't think it is what Mark had in his mind.
Here is patch to remove MCOUNT_ADDR from all arch, but I think it isn't as good as current MCOUNT_ADDR. So how to remove the MCOUNT_ADDR depedency? Could we
patch ftrace_make_nop to accept one more parameter? I.E int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
unsigned long addr, bool init)
or
make "0" a special meaning to tell ftrace_make_nop to "init"?
Thanks
<---8
ftrace: introdue ftrace_call_init
On some arch, the FTRACE_WITH_REGS is implemented with gcc's
-fpatchable-function-entry (=2), gcc adds 2 NOPs at the beginning
of each function, so this makes the MCOUNT_ADDR useless. In ftrace common framework, MCOUNT_ADDR is mostly used to "init" the nop, so let's introcude ftrace_call_init() and remove MCOUNT_ADDR.
--- arch/arm/include/asm/ftrace.h | 1 - arch/arm/kernel/ftrace.c | 5 +++++ arch/arm64/include/asm/ftrace.h | 1 - arch/arm64/kernel/ftrace.c | 5 +++++ arch/csky/kernel/ftrace.c | 5 +++++ arch/ia64/include/asm/ftrace.h | 2 -- arch/ia64/kernel/ftrace.c | 5 +++++ arch/microblaze/include/asm/ftrace.h | 1 - arch/microblaze/kernel/ftrace.c | 5 +++++ arch/mips/include/asm/ftrace.h | 1 - arch/mips/kernel/ftrace.c | 9 +++++++-- arch/nds32/include/asm/ftrace.h | 1 - arch/nds32/kernel/ftrace.c | 5 +++++ arch/parisc/include/asm/ftrace.h | 1 - arch/parisc/kernel/ftrace.c | 5 +++++ arch/powerpc/include/asm/ftrace.h | 1 - arch/powerpc/kernel/trace/ftrace.c | 5 +++++ arch/riscv/include/asm/ftrace.h | 1 - arch/riscv/kernel/ftrace.c | 5 +++++ arch/s390/include/asm/ftrace.h | 1 - arch/s390/kernel/ftrace.c | 22 +++++++++++++++++----- arch/sh/include/asm/ftrace.h | 2 -- arch/sh/kernel/ftrace.c | 5 +++++ arch/sparc/include/asm/ftrace.h | 1 - arch/sparc/kernel/ftrace.c | 5 +++++ arch/x86/include/asm/ftrace.h | 1 - arch/x86/kernel/ftrace.c | 28 +++++++++++++++------------- arch/xtensa/include/asm/ftrace.h | 1 - include/linux/ftrace.h | 1 + kernel/trace/ftrace.c | 2 +- 30 files changed, 96 insertions(+), 37 deletions(-)
arch/arm/include/asm/ftrace.h +0 -1
@@ -7,7 +7,6 @@#endif #ifdef CONFIG_FUNCTION_TRACER -#define MCOUNT_ADDR ((unsigned long)(__gnu_mcount_nc))#define MCOUNT_INSN_SIZE 4/* sizeof mcount call */ #ifndef __ASSEMBLY__
arch/arm/kernel/ftrace.c +5 -0
@@ -172,6 +172,11 @@ int ftrace_make_nop(struct module *mod,return ret; } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )__gnu_mcount_nc); +} +int __init ftrace_dyn_arch_init (void ) {return 0 ;
arch/arm64/include/asm/ftrace.h +0 -1
@@ -11,7 +11,6 @@#include <asm/insn.h> #define HAVE_FUNCTION_GRAPH_FP_TEST -#define MCOUNT_ADDR ((unsigned long)_mcount)#define MCOUNT_INSN_SIZE AARCH64_INSN_SIZE#ifndef __ASSEMBLY__
arch/arm64/kernel/ftrace.c +5 -0
@@ -196,6 +196,11 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,return ftrace_modify_code (pc, old, new, validate); } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )_mcount); +} +void arch_ftrace_update_code (int command) { command |=FTRACE_MAY_SLEEP ;
arch/csky/kernel/ftrace.c +5 -0
@@ -122,6 +122,11 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,return ftrace_modify_code (rec->ip, addr, false, false); } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )_mcount); +} +int ftrace_update_ftrace_func (ftrace_func_t func) { int ret = ftrace_modify_code((unsigned long)&ftrace_call,
arch/ia64/include/asm/ftrace.h +0 -2
@@ -9,8 +9,6 @@extern void _mcount (unsigned long pfs,unsigned long r1,unsigned long b0,unsigned long r0);#define mcount _mcount -/* In IA64, MCOUNT_ADDR is set in link time, so it's not a constant at compile time */ -#define MCOUNT_ADDR (((struct fnptr *)mcount)->ip)#define FTRACE_ADDR (((struct fnptr *)ftrace_caller)->ip)static inline unsigned long ftrace_call_adjust (unsigned long addr)
arch/ia64/kernel/ftrace.c +5 -0
@@ -169,6 +169,11 @@ int ftrace_make_nop(struct module *mod,return ftrace_modify_code (rec->ip,NULL , new,0 ); } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, ((struct fnptr *)mcount)->ip); +} +int ftrace_make_call (struct dyn_ftrace *rec,unsigned long addr) {unsigned long ip = rec->ip;
arch/microblaze/include/asm/ftrace.h +0 -1
@@ -4,7 +4,6 @@#ifdef CONFIG_FUNCTION_TRACER -#define MCOUNT_ADDR ((unsigned long)(_mcount))#define MCOUNT_INSN_SIZE 8/* sizeof mcount call */ #ifndef __ASSEMBLY__
arch/microblaze/kernel/ftrace.c +5 -0
@@ -149,6 +149,11 @@ int ftrace_make_nop(struct module *mod,return ret; } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )_mcount); +} +/* I believe that first is called ftrace_make_nop before this function */ int ftrace_make_call (struct dyn_ftrace *rec,unsigned long addr) {
arch/mips/include/asm/ftrace.h +0 -1
@@ -12,7 +12,6 @@#ifdef CONFIG_FUNCTION_TRACER -#define MCOUNT_ADDR ((unsigned long)(_mcount))#define MCOUNT_INSN_SIZE 4/* sizeof mcount call */ #ifndef __ASSEMBLY__
arch/mips/kernel/ftrace.c +7 -2
@@ -61,7 +61,7 @@ static inline void ftrace_dyn_arch_init_insns(void)/* la v1, _mcount */ v1 =3 ; buf = (u32 *)&insn_la_mcount[0 ]; -UASM_i_LA (&buf, v1,MCOUNT_ADDR ); +UASM_i_LA (&buf, v1, (unsigned long )_mcount);/* jal (ftrace_caller + 8), jump over the first two instruction */ buf = (u32 *)&insn_jal_ftrace_caller;
@@ -200,6 +200,11 @@ int ftrace_make_nop(struct module *mod,#endif } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )_mcount); +} +int ftrace_make_call (struct dyn_ftrace *rec,unsigned long addr) {unsigned int new;
@@ -232,7 +237,7 @@ int __init ftrace_dyn_arch_init(void)ftrace_dyn_arch_init_insns ();/* Remove "b ftrace_stub" to ensure ftrace_caller() is executed */ -ftrace_modify_code (MCOUNT_ADDR ,INSN_NOP ); +ftrace_modify_code ((unsigned long )_mcount,INSN_NOP );return 0 ; }
arch/nds32/include/asm/ftrace.h +0 -1
@@ -7,7 +7,6 @@#define HAVE_FUNCTION_GRAPH_FP_TEST -#define MCOUNT_ADDR ((unsigned long)(_mcount))/* mcount call is composed of three instructions: * sethi + ori + jral */
arch/nds32/kernel/ftrace.c +5 -0
@@ -203,6 +203,11 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, return ftrace_modify_code(pc, call_insn, nop_insn, true); } + +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )_mcount); +}#endif /* CONFIG_DYNAMIC_FTRACE */ #ifdef CONFIG_FUNCTION_GRAPH_TRACER
arch/parisc/include/asm/ftrace.h +0 -1
@@ -5,7 +5,6 @@#ifndef __ASSEMBLY__extern void mcount (void ); -#define MCOUNT_ADDR ((unsigned long)mcount)#define MCOUNT_INSN_SIZE 4#define CC_USING_NOP_MCOUNT extern unsigned long sys_call_table[];
arch/parisc/kernel/ftrace.c +5 -0
@@ -186,4 +186,9 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, insn, sizeof(insn)-4); return 0; } + +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )mcount); +}#endif
arch/powerpc/include/asm/ftrace.h +0 -1
@@ -5,7 +5,6 @@#include <asm/types.h> #ifdef CONFIG_FUNCTION_TRACER -#define MCOUNT_ADDR ((unsigned long)(_mcount))#define MCOUNT_INSN_SIZE 4/* sizeof mcount call */ #ifdef __ASSEMBLY__
arch/powerpc/kernel/trace/ftrace.c +5 -0
@@ -472,6 +472,11 @@ int ftrace_make_nop(struct module *mod,#endif /* CONFIG_MODULES */ } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )_mcount); +} +#ifdef CONFIG_MODULES #ifdef CONFIG_PPC64 /*
arch/riscv/include/asm/ftrace.h +0 -1
@@ -33,7 +33,6 @@ struct dyn_arch_ftrace {* both auipc and jalr at the same time. */ -#define MCOUNT_ADDR ((unsigned long)_mcount)#define JALR_SIGN_MASK (0x00000800)#define JALR_OFFSET_MASK (0x00000fff)#define AUIPC_OFFSET_MASK (0xfffff000)
arch/riscv/kernel/ftrace.c +5 -0
@@ -88,6 +88,11 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,return __ftrace_modify_call (rec->ip, addr, false); } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )_mcount); +} +int ftrace_update_ftrace_func (ftrace_func_t func) { int ret = __ftrace_modify_call((unsigned long)&ftrace_call,
arch/s390/include/asm/ftrace.h +0 -1
@@ -30,7 +30,6 @@ extern unsigned long ftrace_plt;struct dyn_arch_ftrace { }; -#define MCOUNT_ADDR ((unsigned long)_mcount)#define FTRACE_ADDR ((unsigned long)ftrace_caller)#define KPROBE_ON_FTRACE_NOP 0
arch/s390/kernel/ftrace.c +17 -5
@@ -110,11 +110,7 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,if (probe_kernel_read (&old, (void *) rec->ip,sizeof (old)))return -EFAULT ; -if (addr ==MCOUNT_ADDR ) { -/* Initial code replacement */ -ftrace_generate_orig_insn (&orig); -ftrace_generate_nop_insn (&new); - }else if (is_kprobe_on_ftrace (&old)) { +if (is_kprobe_on_ftrace (&old)) {/* * If we find a breakpoint instruction, a kprobe has been * placed at the beginning of the function. We write the
@@ -136,6 +132,22 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,return 0 ; } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +struct ftrace_insn orig, new, old; + +if (probe_kernel_read (&old, (void *) rec->ip,sizeof (old))) +return -EFAULT ; +ftrace_generate_orig_insn (&orig); +ftrace_generate_nop_insn (&new); +/* Verify that the to be replaced code matches what we expect. */ +if (memcmp (&orig, &old,sizeof (old))) +return -EINVAL ; +s390_kernel_write ((void *) rec->ip, &new,sizeof (new)); + +return 0 ; +} +int ftrace_make_call (struct dyn_ftrace *rec,unsigned long addr) {struct ftrace_insn orig, new, old;
arch/sh/include/asm/ftrace.h +0 -2
@@ -10,8 +10,6 @@#ifndef __ASSEMBLY__extern void mcount (void ); -#define MCOUNT_ADDR ((unsigned long)(mcount)) -#ifdef CONFIG_DYNAMIC_FTRACE #define CALL_ADDR ((long)(ftrace_call))#define STUB_ADDR ((long)(ftrace_stub))
arch/sh/kernel/ftrace.c +5 -0
@@ -242,6 +242,11 @@ int ftrace_make_nop(struct module *mod,return ftrace_modify_code (rec->ip, old, new); } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )mcount); +} +int ftrace_make_call (struct dyn_ftrace *rec,unsigned long addr) {unsigned char *new, *old;
arch/sparc/include/asm/ftrace.h +0 -1
@@ -3,7 +3,6 @@#define _ASM_SPARC64_FTRACE#ifdef CONFIG_MCOUNT -#define MCOUNT_ADDR ((unsigned long)(_mcount))#define MCOUNT_INSN_SIZE 4/* sizeof mcount call */ #ifndef __ASSEMBLY__
arch/sparc/kernel/ftrace.c +5 -0
@@ -63,6 +63,11 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long adreturn ftrace_modify_code (ip, old, new); } +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) +{ +return ftrace_make_nop (mod, rec, (unsigned long )_mcount); +} +int ftrace_make_call (struct dyn_ftrace *rec,unsigned long addr) {unsigned long ip = rec->ip;
arch/x86/include/asm/ftrace.h +0 -1
@@ -6,7 +6,6 @@#ifndef CC_USING_FENTRY # error Compiler does not support fentry?#endif -# define MCOUNT_ADDR ((unsigned long)(__fentry__))#define MCOUNT_INSN_SIZE 5/* sizeof mcount call */ #ifdef CONFIG_DYNAMIC_FTRACE
arch/x86/kernel/ftrace.c +15 -13
@@ -149,26 +149,28 @@ ftrace_modify_code_direct(unsigned long ip, unsigned const char *old_code,return 0 ; } -int ftrace_make_nop (struct module *mod, -struct dyn_ftrace *rec,unsigned long addr) +/* +* On boot up, and when modules are loaded, the __fentry__ +* is converted to a nop, and will never become __fentry__ +* again. This code is either running before SMP (on boot up) +* or before the code will ever be executed (module load). +* We do not want to use the breakpoint version in this case, +* just modify the code directly. +*/ +int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec) {unsigned const char *new, *old;unsigned long ip = rec->ip; - old =ftrace_call_replace (ip, addr); + old =ftrace_call_replace (ip, (unsigned long )__fentry__); new =ftrace_nop_replace (); -/* -* On boot up, and when modules are loaded, the MCOUNT_ADDR -* is converted to a nop, and will never become MCOUNT_ADDR -* again. This code is either running before SMP (on boot up) -* or before the code will ever be executed (module load). -* We do not want to use the breakpoint version in this case, -* just modify the code directly. -*/ -if (addr ==MCOUNT_ADDR ) -return ftrace_modify_code_direct (rec->ip, old, new); +return ftrace_modify_code_direct (ip, old, new); +} +int ftrace_make_nop (struct module *mod, +struct dyn_ftrace *rec,unsigned long addr) +{ ftrace_expected =NULL ;/* Normal cases use add_brk_on_nop */
arch/xtensa/include/asm/ftrace.h +0 -1
@@ -28,7 +28,6 @@ extern unsigned long return_address(unsigned level);#ifdef CONFIG_FUNCTION_TRACER -#define MCOUNT_ADDR ((unsigned long)(_mcount))#define MCOUNT_INSN_SIZE 3#ifndef __ASSEMBLY__
include/linux/ftrace.h +1 -0
@@ -458,6 +458,7 @@ extern void ftrace_regs_caller(void);extern void ftrace_call (void );extern void ftrace_regs_call (void );extern void mcount_call (void ); +extern int ftrace_call_init (struct module *mod,struct dyn_ftrace *rec);void ftrace_modify_all_code (int command);
kernel/trace/ftrace.c +1 -1
@@ -2500,7 +2500,7 @@ ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec)if (unlikely (ftrace_disabled))return 0 ; - ret =ftrace_make_nop (mod, rec,MCOUNT_ADDR ); + ret =ftrace_call_init (mod, rec);if (ret) { ftrace_bug_type =FTRACE_BUG_INIT ;ftrace_bug (ret, rec);
Signature · 1 line
2.23.0.rc1
PATCH 2/3 arm64: implement ftrace with regs
In reply to Jisheng ZhangFrom: Torsten Duwe <[email protected]>
Implement ftrace with regs, based on the new gcc flag -fpatchable-function-entry (=2)
Now that gcc8 added 2 NOPs at the beginning of each function, replace the first NOP thus generated with a quick LR saver (move it to scratch reg x9), so the 2nd replacement insn, the call to ftrace, does not clobber the value. Ftrace will then generate the standard stack frames.
Note that patchable-function-entry in GCC disables IPA-RA, which means ABI register calling conventions are obeyed and scratch registers such as x9 are available.
Introduce and handle an ftrace_regs_trampoline for module PLTs, right after ftrace_trampoline in an ftrace_trampolines[2] array, and double the size of the corresponding special section.
--- arch/arm64/include/asm/ftrace.h | 12 ++- arch/arm64/include/asm/module.h | 3 +- arch/arm64/kernel/entry-ftrace.S | 125 ++++++++++++++++++++++++++-- arch/arm64/kernel/ftrace.c | 138 +++++++++++++++++++++++-------- arch/arm64/kernel/module-plts.c | 3 +- arch/arm64/kernel/module.c | 2 +- 6 files changed, 239 insertions(+), 44 deletions(-)
arch/arm64/include/asm/ftrace.h +11 -1
@@ -11,9 +11,12 @@#include <asm/insn.h> #define HAVE_FUNCTION_GRAPH_FP_TEST -#define MCOUNT_ADDR ((unsigned long)_mcount)#define MCOUNT_INSN_SIZE AARCH64_INSN_SIZE +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +#define ARCH_SUPPORTS_FTRACE_OPS 1 +#endif +#ifndef __ASSEMBLY__#include <linux/compat.h>
@@ -30,6 +33,13 @@ extern void return_to_handler(void);static inline unsigned long ftrace_call_adjust (unsigned long addr) { +/* +* For -fpatchable-function-entry=2, there's first the +* LR saver, and only then the actual call insn. +* Advance addr accordingly. +*/ +if (IS_ENABLED (CONFIG_DYNAMIC_FTRACE_WITH_REGS )) +return (addr +AARCH64_INSN_SIZE );/* * addr is the address of the mcount call instruction. * recordmcount does the necessary offset calculation.
arch/arm64/include/asm/module.h +2 -1
@@ -21,7 +21,8 @@ struct mod_arch_specific {struct mod_plt_sec init;/* for CONFIG_DYNAMIC_FTRACE */ - struct plt_entry *ftrace_trampoline; +struct plt_entry *ftrace_trampolines; +#define MOD_ARCH_NR_FTRACE_TRAMPOLINES 2 };#endif
arch/arm64/kernel/entry-ftrace.S +120 -5
150 lines not on this page · raw message
arch/arm64/kernel/ftrace.c +103 -35
169 lines not on this page · raw message
arch/arm64/kernel/module-plts.c +2 -1
@@ -330,7 +330,8 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs, tramp->sh_type =SHT_NOBITS ; tramp->sh_flags =SHF_EXECINSTR |SHF_ALLOC ; tramp->sh_addralign = __alignof__(struct plt_entry ); - tramp->sh_size =sizeof (struct plt_entry ); + tramp->sh_size =MOD_ARCH_NR_FTRACE_TRAMPOLINES + *sizeof (struct plt_entry ); }return 0 ;
arch/arm64/kernel/module.c +1 -1
@@ -483,7 +483,7 @@ int module_finalize(const Elf_Ehdr *hdr,#ifdef CONFIG_ARM64_MODULE_PLTS if (IS_ENABLED (CONFIG_DYNAMIC_FTRACE ) && !strcmp (".text.ftrace_trampoline" , secstrs + s->sh_name)) - me->arch.ftrace_trampoline = (void *)s->sh_addr; + me->arch.ftrace_trampolines = (void *)s->sh_addr;#endif }
Signature · 7 lines
2.23.0.rc1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 3/3 arm64: use -fpatchable-function-entry if available
In reply to Jisheng ZhangFrom: Torsten Duwe <[email protected]>
Test whether gcc supports -fpatchable-function-entry and use it to promote DYNAMIC_FTRACE to DYNAMIC_FTRACE_WITH_REGS. Amend support for the new object section that holds the locations (__patchable_function_entries) and define a proper "notrace" attribute to switch it off.
--- arch/arm64/Kconfig | 2 ++ arch/arm64/Makefile | 5 +++++ kernel/module.c | 7 ++++++- 3 files changed, 13 insertions(+), 1 deletion(-)
arch/arm64/Kconfig +2 -0
@@ -144,6 +144,8 @@ config ARM64 select HAVE_DEBUG_KMEMLEAK select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE + select HAVE_DYNAMIC_FTRACE_WITH_REGS \ + if $(cc-option,-fpatchable-function-entry=2) select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_FAST_GUP select HAVE_FTRACE_MCOUNT_RECORD
arch/arm64/Makefile +5 -0
@@ -104,6 +104,11 @@ ifeq ($(CONFIG_ARM64_MODULE_PLTS),y) KBUILD_LDFLAGS_MODULE += -T $(srctree)/arch/arm64/kernel/module.lds endif +ifeq ($(CONFIG_DYNAMIC_FTRACE_WITH_REGS),y) + KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY + CC_FLAGS_FTRACE := -fpatchable-function-entry=2 +endif + # Default value head-y := arch/arm64/kernel/head.o
kernel/module.c +6 -1
@@ -3136,7 +3136,12 @@ static int find_module_sections(struct module *mod, struct load_info *info)#endif #ifdef CONFIG_FTRACE_MCOUNT_RECORD /* sechdrs[0].sh_size is always zero */ - mod->ftrace_callsites =section_objs (info,"__mcount_loc" , + mod->ftrace_callsites =section_objs (info, +#ifdef CC_USING_PATCHABLE_FUNCTION_ENTRY +"__patchable_function_entries" , +#else +"__mcount_loc" , +#endifsizeof (*mod->ftrace_callsites), &mod->num_ftrace_callsites);#endif
Signature · 7 lines
2.23.0.rc1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel