arm: ftrace: Adds support for CONFIG_DYNAMIC_FTRACE_WITH_REGS
Subsystems (2)
PATCH v5 1/1 arm: ftrace: Adds support for CONFIG_DYNAMIC_FTRACE_WITH_REGS
The DYNAMIC_FTRACE_WITH_REGS configuration makes it possible for a ftrace operation to specify if registers need to saved/restored by the ftrace handler. This is needed by kgraft and possibly other ftrace-based tools, and the ARM architecture is currently lacking this feature. It would also be the first step to support the "Kprobes-on-ftrace" optimization on ARM.
This patch introduces a new ftrace handler that stores the registers on the stack before calling the next stage. The registers are restored from the stack before going back to the instrumented function.
A side-effect of this patch is to activate the support for ftrace_modify_call() as it defines ARCH_SUPPORTS_FTRACE_OPS for the ARM architecture.
--- arch/arm/Kconfig | 1 + arch/arm/include/asm/ftrace.h | 4 ++ arch/arm/kernel/entry-ftrace.S | 83 ++++++++++++++++++++++++++++++++++++++++++ arch/arm/kernel/ftrace.c | 48 +++++++++++++++++++++--- 4 files changed, 131 insertions(+), 5 deletions(-)
arch/arm/Kconfig +1 -0
@@ -56,6 +56,7 @@ config ARM select HAVE_DMA_API_DEBUG select HAVE_DMA_CONTIGUOUS if MMU select HAVE_DYNAMIC_FTRACE if (!XIP_KERNEL) && !CPU_ENDIAN_BE32 && MMU + select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU select HAVE_EXIT_THREAD select HAVE_FTRACE_MCOUNT_RECORD if (!XIP_KERNEL)
arch/arm/include/asm/ftrace.h +4 -0
@@ -1,6 +1,10 @@#ifndef _ASM_ARM_FTRACE#define _ASM_ARM_FTRACE +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +#define ARCH_SUPPORTS_FTRACE_OPS 1 +#endif +#ifdef CONFIG_FUNCTION_TRACER #define MCOUNT_ADDR ((unsigned long)(__gnu_mcount_nc))#define MCOUNT_INSN_SIZE 4/* sizeof mcount call */
arch/arm/kernel/entry-ftrace.S +83 -0
107 lines not on this page · raw message
arch/arm/kernel/ftrace.c +43 -5
89 lines not on this page · raw message
Signature · 1 line
2.7.4
Quoted 7 lines · Abel Vesa
arch/arm/kernel/entry-ftrace.S +0
-0@@ -92,12 +92,78 @@ 2: mcount_exit .endm
arch/arm/kernel/entry-ftrace.S line 103
Quoted 10 lines
arch/arm/kernel/entry-ftrace.S +9
-0+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS + +.macro __ftrace_regs_caller + + sub sp, sp, #8 @ space for CPSR and OLD_R0 (not used) + + add ip, sp, #12 @ move in IP the value of SP as it was + @ before the push {lr} of the mcount mechanism + stmdb sp!, {ip,lr,pc}
This doesn't build with CONFIG_THUMB2_KERNEL:
entry-ftrace.S:285: Error: PC not allowed in register list -- `stmdb sp!,{ip,lr,pc}'
Saving PC in STMDB is prohibited in Thumb and deprecated in ARM.
arch/arm/kernel/entry-ftrace.S line 114
Quoted 11 lines
arch/arm/kernel/entry-ftrace.S +11
-0+ stmdb sp!, {r0-r11,lr} + + ldr r0, [sp, #S_LR] @ replace PC with LR + str r0, [sp, #S_PC] @ into pt_regs + + ldr r1, [sp, #PT_REGS_SIZE] @ replace new LR with + str r1, [sp, #S_LR] @ previous LR into pt_regs + + @ stack content at this point: + @ 0 4 48 52 56 60 64 68 72 + @ R0 | R1 | ... | LR | SP + 4 | previous LR | LR | PSR | OLD_R0 | previous LR |
So this code only adjust the SP by 72 bytes. Since the stack is not aligned to 8 bytes at entry to mcount, this code calls into C functions with a misaligned stack. This is a violation of the procedure call standard.
Also, I believe this should have unwind annotations to indicate where it saved the registers. See mcount_enter for an example.
Quoted 9 lines
arch/arm/kernel/entry-ftrace.S +9
-0+ + mov r3, sp @ struct pt_regs* + ldr r2, =function_trace_op + ldr r2, [r2] @ pointer to the current + @ function tracing op + ldr r1, [sp, #PT_REGS_SIZE] @ lr of instrumented func + mcount_adjust_addr r0, lr @ instrumented function + + .globl ftrace_regs_call
arch/arm/kernel/entry-ftrace.S line 133
Quoted 10 lines
arch/arm/kernel/entry-ftrace.S +10
-0+ftrace_regs_call: + bl ftrace_stub + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + .globl ftrace_graph_regs_call +ftrace_graph_regs_call: + mov r0, r0 +#endif + @ pop saved regs + ldmia sp, {r0-r15}
This doesn't build either:
entry-ftrace.S:285: Error: LR and PC should not both be in register list -- `ldmia sp,{r0-r15}'
Restoring LR and PC together is prohibited in Thumb and deprecated in ARM. It's the same case with SP.
Quoted 6 lines
arch/arm/kernel/entry-ftrace.S +6
-0+.endm + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +.macro __ftrace_graph_regs_caller + + sub r0, fp, #4 @ lr of instrumented routine (parent)
arch/arm/kernel/entry-ftrace.S line 149
Quoted 10 lines
arch/arm/kernel/entry-ftrace.S +10
-0+ + @ called from __ftrace_regs_caller + ldr r1, [sp, #S_PC] @ instrumented routine (func) + mcount_adjust_addr r1, r1 + + mov r2, fp @ frame pointer + bl prepare_ftrace_return + + @ pop registers saved in ftrace_regs_caller + ldmia sp, {r0-r15}
This doesn't get built on CONFIG_THUMB2_KERNEL since there's no suppport for CONFIG_FUNCTION_GRAPH_TRACER there, but this is also using deprecated operands.
Quoted 3 lines
arch/arm/kernel/entry-ftrace.S +3
-0+.endm +#endif +#endif
...
Quoted 17 lines
arch/arm/kernel/ftrace.c +0
-5@@ -29,11 +29,6 @@#endif #ifdef CONFIG_DYNAMIC_FTRACE -#ifdef CONFIG_OLD_MCOUNT -#define OLD_MCOUNT_ADDR ((unsigned long) mcount) -#define OLD_FTRACE_ADDR ((unsigned long) ftrace_caller_old) - -#define OLD_NOP 0xe1a00000 /* mov r0, r0 */static int __ftrace_modify_code (void *data) {
arch/arm/kernel/ftrace.c void arch_ftrace_update_code(int command) · line 55
Quoted 10 lines
arch/arm/kernel/ftrace.c +6
-0@@ -51,6 +46,12 @@ void arch_ftrace_update_code(int command)stop_machine (__ftrace_modify_code, &command,NULL ); } +#ifdef CONFIG_OLD_MCOUNT +#define OLD_MCOUNT_ADDR ((unsigned long) mcount) +#define OLD_FTRACE_ADDR ((unsigned long) ftrace_caller_old) + +#define OLD_NOP 0xe1a00000/* mov r0, r0 */ +static unsigned long ftrace_nop_replace (struct dyn_ftrace *rec)
This chunk above fixes the problem which Nicolai pointed out earlier. Since it's a bug fix for the current dynamic ftrace code, it should really be submitted as a separate patch. Feel free to use some variant of this commit message which I wrote when I ran into this yesterday:
ARM: ftrace: fix dynamic ftrace with DEBUG_RODATA and !FRAME_POINTER
The support for dynamic ftrace with CONFIG_DEBUG_RODATA involves overriding the weak arch_ftrace_update_code() with a variant which makes the kernel text writable around the patching.
This override was however added under the CONFIG_OLD_MCOUNT ifdef, and CONFIG_OLD_MCOUNT is only enabled if frame pointers are enabled.
This leads to non-functional dynamic ftrace (ftrace triggers a WARN_ON()) when CONFIG_DEBUG_RODATA is enabled and CONFIG_FRAME_POINTER is not.
Move the override out of that ifdef and into the CONFIG_DYNAMIC_FTRACE ifdef where it belongs.
arch/arm/kernel/ftrace.c int ftrace_update_ftrace_func(ftrace_func_t func) · line 143
Quoted 7 lines
arch/arm/kernel/ftrace.c +1
-0{return rec->arch.old_mcount ?OLD_NOP :NOP ;@@ -139,6 +140,15 @@ int ftrace_update_ftrace_func(ftrace_func_t func) ret =ftrace_modify_code (pc,0 , new, false); +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
This code (and the rest of your additions to this file below), are already under CONFIG_DYNAMIC_FTRACE and afaics CONFIG_DYNAMIC_FTRACE_WITH_REGS is always enabled when the former is enabled, so could you please drop these ifdefs?
The code in entry-ftrace.S under CONFIG_DYNAMIC_FTRACE_WITH_REGS would also benefit from just using CONFIG_DYNAMIC_FTRACE everywhere instead.
Quoted 67 lines · Rabin Vincent
Quoted 7 lines · Abel Vesa
arch/arm/kernel/entry-ftrace.S +0
-0@@ -92,12 +92,78 @@ 2: mcount_exit .endmarch/arm/kernel/entry-ftrace.S line 103
Quoted 10 lines
arch/arm/kernel/entry-ftrace.S +9
-0+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS + +.macro __ftrace_regs_caller + + sub sp, sp, #8 @ space for CPSR and OLD_R0 (not used) + + add ip, sp, #12 @ move in IP the value of SP as it was + @ before the push {lr} of the mcount mechanism + stmdb sp!, {ip,lr,pc}This doesn't build with
CONFIG_THUMB2_KERNEL:entry-ftrace.S:285: Error: PC not allowed in register list -- `stmdb sp!,{ip,lr,pc}'
Saving PC in STMDB is prohibited in Thumb and deprecated in ARM.
arch/arm/kernel/entry-ftrace.S line 114
Quoted 11 lines
arch/arm/kernel/entry-ftrace.S +11
-0+ stmdb sp!, {r0-r11,lr} + + ldr r0, [sp, #S_LR] @ replace PC with LR + str r0, [sp, #S_PC] @ into pt_regs + + ldr r1, [sp, #PT_REGS_SIZE] @ replace new LR with + str r1, [sp, #S_LR] @ previous LR into pt_regs + + @ stack content at this point: + @ 0 4 48 52 56 60 64 68 72 + @ R0 | R1 | ... | LR | SP + 4 | previous LR | LR | PSR | OLD_R0 | previous LR |So this code only adjust the SP by 72 bytes. Since the stack is not aligned to 8 bytes at entry to mcount, this code calls into C functions with a misaligned stack. This is a violation of the procedure call standard.
Also, I believe this should have unwind annotations to indicate where it saved the registers. See
mcount_enterfor an example.Quoted 9 lines
arch/arm/kernel/entry-ftrace.S +9
-0+ + mov r3, sp @ struct pt_regs* + ldr r2, =function_trace_op + ldr r2, [r2] @ pointer to the current + @ function tracing op + ldr r1, [sp, #PT_REGS_SIZE] @ lr of instrumented func + mcount_adjust_addr r0, lr @ instrumented function + + .globl ftrace_regs_callarch/arm/kernel/entry-ftrace.S line 133
Quoted 10 lines
arch/arm/kernel/entry-ftrace.S +10
-0+ftrace_regs_call: + bl ftrace_stub + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + .globl ftrace_graph_regs_call +ftrace_graph_regs_call: + mov r0, r0 +#endif + @ pop saved regs + ldmia sp, {r0-r15}This doesn't build either:
Quoted 5 lines
entry-ftrace.S:285: Error: LR and PC should not both be in register list -- `ldmia sp,{r0-r15}'
Restoring LR and PC together is prohibited in Thumb and deprecated in ARM. It's the same case with SP.
We could do it in separate instructions, something like:
@ pop saved regs
ldmia sp!, {r0-r12}
ldr ip, [sp, #8]
ldr lr, [sp, #4]
ldr sp, [sp, #0]
mov pc, ip Same goes for saving the regs, except I need to replace the LR stored outside of ftrace_regs_caller with OLD_R0 so that the stack would become aligned to 8 bytes.
Quoted 67 lines
Quoted 6 lines
arch/arm/kernel/entry-ftrace.S +6
-0+.endm + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +.macro __ftrace_graph_regs_caller + + sub r0, fp, #4 @ lr of instrumented routine (parent)arch/arm/kernel/entry-ftrace.S line 149
Quoted 10 lines
arch/arm/kernel/entry-ftrace.S +10
-0+ + @ called from __ftrace_regs_caller + ldr r1, [sp, #S_PC] @ instrumented routine (func) + mcount_adjust_addr r1, r1 + + mov r2, fp @ frame pointer + bl prepare_ftrace_return + + @ pop registers saved in ftrace_regs_caller + ldmia sp, {r0-r15}This doesn't get built on
CONFIG_THUMB2_KERNELsince there's no suppport forCONFIG_FUNCTION_GRAPH_TRACERthere, but this is also using deprecated operands.Quoted 3 lines
arch/arm/kernel/entry-ftrace.S +3
-0+.endm +#endif +#endif...
Quoted 17 lines
arch/arm/kernel/ftrace.c +0
-5@@ -29,11 +29,6 @@#endif #ifdef CONFIG_DYNAMIC_FTRACE -#ifdef CONFIG_OLD_MCOUNT -#define OLD_MCOUNT_ADDR ((unsigned long) mcount) -#define OLD_FTRACE_ADDR ((unsigned long) ftrace_caller_old) - -#define OLD_NOP 0xe1a00000 /* mov r0, r0 */static int __ftrace_modify_code (void *data) {arch/arm/kernel/ftrace.c void arch_ftrace_update_code(int command) · line 55
Quoted 10 lines
arch/arm/kernel/ftrace.c +6
-0@@ -51,6 +46,12 @@ void arch_ftrace_update_code(int command)stop_machine (__ftrace_modify_code, &command,NULL ); } +#ifdef CONFIG_OLD_MCOUNT +#define OLD_MCOUNT_ADDR ((unsigned long) mcount) +#define OLD_FTRACE_ADDR ((unsigned long) ftrace_caller_old) + +#define OLD_NOP 0xe1a00000/* mov r0, r0 */ +static unsigned long ftrace_nop_replace (struct dyn_ftrace *rec)This chunk above fixes the problem which Nicolai pointed out earlier. Since it's a bug fix for the current dynamic ftrace code, it should really be submitted as a separate patch. Feel free to use some variant of this commit message which I wrote when I ran into this yesterday:
ARM: ftrace: fix dynamic ftrace with
DEBUG_RODATAand !FRAME_POINTERThe support for dynamic ftrace with
CONFIG_DEBUG_RODATAinvolves overriding the weakarch_ftrace_update_code()with a variant which makes the kernel text writable around the patching.This override was however added under the
CONFIG_OLD_MCOUNTifdef, andCONFIG_OLD_MCOUNTis only enabled if frame pointers are enabled.
Quoted 9 lines
This leads to non-functional dynamic ftrace (ftrace triggers a
WARN_ON()) whenCONFIG_DEBUG_RODATAis enabled andCONFIG_FRAME_POINTERis not.Move the override out of that ifdef and into the
CONFIG_DYNAMIC_FTRACEifdef where it belongs.Fixes 80d6b0c2eed2a ("ARM: mm: allow text and rodata sections to be read-only")
Sent as a separate patch already.
Quoted 13 lines
arch/arm/kernel/ftrace.c int ftrace_update_ftrace_func(ftrace_func_t func) · line 143
Quoted 7 lines
arch/arm/kernel/ftrace.c +1
-0{return rec->arch.old_mcount ?OLD_NOP :NOP ;@@ -139,6 +140,15 @@ int ftrace_update_ftrace_func(ftrace_func_t func) ret =ftrace_modify_code (pc,0 , new, false); +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS This code (and the rest of your additions to this file below), are already under
CONFIG_DYNAMIC_FTRACEand afaicsCONFIG_DYNAMIC_FTRACE_WITH_REGSis always enabled when the former is enabled, so could you please drop these ifdefs?
Quoted 2 lines
The code in
entry-ftrace.SunderCONFIG_DYNAMIC_FTRACE_WITH_REGSwould also benefit from just usingCONFIG_DYNAMIC_FTRACEeverywhere instead.
I believe this is incorrect, I think it is worth being compiled in only if CONFIG_DYNAMIC_FTRACE_WITH_REGS, but this is dependent on CONFIG_DYNAMIC_FTRACE and the first one is not necessarily enabled if the second one is. Otherwise, what's the point of having two CONFIGs ?