riscv: Optimize function trace
v11 cover riscv: Optimize function trace
Changes in V11:
- append a patch that makes the DIRECT_CALL samples support RV32I in this series fixing the rv32 build failure reported by Palmer
- validated with ftrace boottime selftest and manual sample modules test in qemu-system for RV32I and RV64I
This series optimizes function trace. The first 3 independent patches has been picked in the V7 version of this series, the subsequent version continues the following 4 patches:
select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY [1] (patch 1)
==========================================================
In RISC-V, -fpatchable-function-entry option is used to support dynamic ftrace in this commit afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT"). So recordmcount don't have to be called to create the __mcount_loc section before the vmlinux linking.
Here selects FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY to tell Makefile not to run recordmcount.
Make function graph use ftrace directly [2] (patch 2)
========================================================
In RISC-V architecture, when we enable the ftrace_graph tracer on some functions, the function tracings on other functions will suffer extra graph tracing work. In essence, graph_ops isn't limited by its func_hash due to the global ftrace_graph_[regs]_call label. That should be corrected.
What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function graph use ftrace directly") that uses graph_ops::func function to install return_hooker and makes the function called against its func_hash.
Add WITH_DIRECT_CALLS support [3] (patch 3, 4)
==============================================
This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V. SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included here as the samples for testing DIRECT_CALLS related interface.
First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide register_ftrace_direct[_multi] interfaces allowing user to register the customed trampoline (direct_caller) as the mcount for one or more target functions. And modify_ftrace_direct[_multi] are also provided for modify direct_caller.
At the same time, the samples in ./samples/ftrace/ can be built as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI selected.
Second, to make the direct_caller and the other ftrace hooks (eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary register
are nominated to store the address of direct_caller in ftrace_regs_caller.
After the setting of the address direct_caller by direct_ops->func and the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to by the jr inst.
The series's old changes related these patches
==========================================
Changes in v10:
https://lore.kernel.org/all/[email protected]/
- add Acked-by from Björn Töpel in patch 2 and patch 4
- replace move with mv in patch3
- prettify patch 2/4 with proper tabs
Changes in v9:
https://lore.kernel.org/linux-riscv/[email protected]/
1. add Acked-by from Björn Töpel in patch 1
2. rebase patch2/patch3 on Linux v6.4-rc1
- patch 2: to make the SAVE_ABI_REGS configurable, revert the modification of mcount-dyn.S from commit (45b32b946a97 "riscv:
entry: Consolidate general regs saving/restoring")
- patch 3: to pass the trace_selftest, add the implement of ftrace_stub_direct_tramp from commit (fee86a4ed536 "ftrace:
selftest: remove broken trace_direct_tramp") ; and fixup the context conflict in Kconfig
Changes in v8:
https://lore.kernel.org/linux-riscv/[email protected]/
- Fix incorrect address values in the 4nd patch
- Rebased on v6.3-rc2
Changes in v7:
https://lore.kernel.org/linux-riscv/[email protected]/
- Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
- Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY [1]
- Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT & HAVE_SAMPLE_FTRACE_DIRECT_MULTI
Changes in v6:
https://lore.kernel.org/linux-riscv/[email protected]/
- Replace 8 with MCOUNT_INSN_SIZE
- Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
- Add Evgenii Shatokhin comment
Changes in v5:
https://lore.kernel.org/linux-riscv/[email protected]/
- Sort Kconfig entries in alphabetical order.
Changes in v4:
https://lore.kernel.org/linux-riscv/[email protected]/
- Include [3] for maintenance. [Song Shuai]
Changes in V3:
https://lore.kernel.org/linux-riscv/[email protected]/
- Include [2] for maintenance. [Song Shuai]
[1]: https://lore.kernel.org/linux-riscv/CAAYs2=j3Eak9vU6xbAw0zPuoh00rh8v5C2U3fePkokZFibWs2g@mail.gmail.com/T/#t
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://lore.kernel.org/linux-riscv/[email protected]/
Song Shuai (5):
riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
riscv: ftrace: Add ftrace_graph_func
riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
samples: ftrace: Make the riscv samples support RV32I
arch/riscv/Kconfig | 4 + arch/riscv/include/asm/ftrace.h | 19 +- arch/riscv/kernel/ftrace.c | 30 ++- arch/riscv/kernel/mcount-dyn.S | 200 ++++++++++++++++---- samples/ftrace/ftrace-direct-modify.c | 35 ++++ samples/ftrace/ftrace-direct-multi-modify.c | 41 ++++ samples/ftrace/ftrace-direct-multi.c | 25 +++ samples/ftrace/ftrace-direct-too.c | 28 +++ samples/ftrace/ftrace-direct.c | 24 +++ 9 files changed, 350 insertions(+), 56 deletions(-)
Signature · 1 line
2.20.1
PATCH v11 1/5 riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
In RISC-V, -fpatchable-function-entry option is used to support dynamic ftrace in this commit afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT"). So recordmcount don't have to be called to create the __mcount_loc section before the vmlinux linking.
Here selects FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY to tell Makefile not to run recordmcount.
--- arch/riscv/Kconfig | 1 + 1 file changed, 1 insertion(+)
arch/riscv/Kconfig +1 -0
@@ -59,6 +59,7 @@ config RISCV select COMMON_CLK select CPU_PM if CPU_IDLE || HIBERNATION select EDAC_SUPPORT + select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if DYNAMIC_FTRACE select GENERIC_ARCH_TOPOLOGY select GENERIC_ATOMIC64 if !64BIT select GENERIC_CLOCKEVENTS_BROADCAST if SMP
Signature · 1 line
2.20.1
PATCH v11 2/5 riscv: ftrace: Add ftrace_graph_func
In reply to Song ShuaiHere implements ftrace_graph_func as the function graph tracing function with FTRACE_WITH_REGS defined.
function_graph_func gets the point of the parent IP and the frame pointer from fregs and call prepare_ftrace_return for function graph tracing.
If FTRACE_WITH_REGS isn't defined, the enable/disable helpers of ftrace_graph_[regs]_call are revised for serving only ftrace_graph_call in the !FTRACE_WITH_REGS version ftrace_caller.
--- arch/riscv/include/asm/ftrace.h | 11 +- arch/riscv/kernel/ftrace.c | 30 +++-- arch/riscv/kernel/mcount-dyn.S | 190 +++++++++++++++++++++++++------- 3 files changed, 175 insertions(+), 56 deletions(-)
arch/riscv/include/asm/ftrace.h +10 -1
@@ -107,7 +107,16 @@ do { \struct dyn_ftrace ;int ftrace_init_nop (struct module *mod,struct dyn_ftrace *rec);#define ftrace_init_nop ftrace_init_nop -#endif + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +struct ftrace_ops ; +struct ftrace_regs ; +void ftrace_graph_func (unsigned long ip,unsigned long parent_ip, +struct ftrace_ops *op,struct ftrace_regs *fregs); +#define ftrace_graph_func ftrace_graph_func +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ + +#endif /* __ASSEMBLY__ */ #endif /* CONFIG_DYNAMIC_FTRACE */
arch/riscv/kernel/ftrace.c +13 -17
@@ -178,32 +178,28 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr, }#ifdef CONFIG_DYNAMIC_FTRACE +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +void ftrace_graph_func (unsigned long ip,unsigned long parent_ip, +struct ftrace_ops *op,struct ftrace_regs *fregs) +{ +struct pt_regs *regs =arch_ftrace_get_regs (fregs); +unsigned long *parent = (unsigned long *)®s->ra; + +prepare_ftrace_return (parent, ip,frame_pointer (regs)); +} +#else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ extern void ftrace_graph_call (void ); -extern void ftrace_graph_regs_call (void );int ftrace_enable_ftrace_graph_caller (void ) { -int ret; - - ret =__ftrace_modify_call ((unsigned long )&ftrace_graph_call, - (unsigned long )&prepare_ftrace_return, true, true); -if (ret) -return ret; - -return __ftrace_modify_call ((unsigned long )&ftrace_graph_regs_call, +return __ftrace_modify_call ((unsigned long )&ftrace_graph_call, (unsigned long )&prepare_ftrace_return, true, true); }int ftrace_disable_ftrace_graph_caller (void ) { -int ret; - - ret =__ftrace_modify_call ((unsigned long )&ftrace_graph_call, - (unsigned long )&prepare_ftrace_return, false, true); -if (ret) -return ret; - -return __ftrace_modify_call ((unsigned long )&ftrace_graph_regs_call, +return __ftrace_modify_call ((unsigned long )&ftrace_graph_call, (unsigned long )&prepare_ftrace_return, false, true); } +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ #endif /* CONFIG_DYNAMIC_FTRACE */ #endif /* CONFIG_FUNCTION_GRAPH_TRACER */
arch/riscv/kernel/mcount-dyn.S +152 -38
219 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH v11 3/5 riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
In reply to Song ShuaiThis patch adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide the register_ftrace_direct[_multi] interfaces allowing users to register the customed trampoline (direct_caller) as the mcount for one or more target functions. And modify_ftrace_direct[_multi] are also provided for modifying direct_caller.
To make the direct_caller and the other ftrace hooks (eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary register is nominated to store the address of direct_caller in ftrace_regs_caller. After the setting of the address direct_caller by direct_ops->func and the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to by the jr inst.
--- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/ftrace.h | 8 ++++++++ arch/riscv/kernel/mcount-dyn.S | 10 ++++++++++ 3 files changed, 19 insertions(+)
arch/riscv/Kconfig +1 -0
@@ -145,6 +145,7 @@ config RISCV select UACCESS_MEMCPY if !MMU select ZONE_DMA32 if 64BIT select HAVE_DYNAMIC_FTRACE if !XIP_KERNEL && MMU && (CLANG_SUPPORTS_DYNAMIC_FTRACE || GCC_SUPPORTS_DYNAMIC_FTRACE) + select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER
arch/riscv/include/asm/ftrace.h +8 -0
@@ -114,6 +114,14 @@ struct ftrace_regs;void ftrace_graph_func (unsigned long ip,unsigned long parent_ip,struct ftrace_ops *op,struct ftrace_regs *fregs);#define ftrace_graph_func ftrace_graph_func + +static inline void +__arch_ftrace_set_direct_caller (struct pt_regs *regs,unsigned long addr) +{ + regs->t1 = addr; +} +#define arch_ftrace_set_direct_caller (fregs, addr) \ + __arch_ftrace_set_direct_caller(&(fregs)->regs, addr)#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ #endif /* __ASSEMBLY__ */
arch/riscv/kernel/mcount-dyn.S +10 -0
@@ -231,6 +231,7 @@ ENDPROC(ftrace_caller) #else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ ENTRY(ftrace_regs_caller) + mv t1, zero SAVE_ABI_REGS 1 PREPARE_ARGS
@@ -239,7 +240,10 @@ ftrace_regs_call: call ftrace_stub RESTORE_ABI_REGS 1 + bnez t1,.Ldirect jr t0 +.Ldirect: + jr t1 ENDPROC(ftrace_regs_caller) ENTRY(ftrace_caller)
@@ -254,3 +258,9 @@ ftrace_call: jr t0 ENDPROC(ftrace_caller) #endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS +SYM_CODE_START(ftrace_stub_direct_tramp) + jr t0 +SYM_CODE_END(ftrace_stub_direct_tramp) +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */
Signature · 1 line
2.20.1
PATCH v11 4/5 samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
In reply to Song Shuaiselect HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for the ftrace-direct*.c files in samples/ftrace/.
--- arch/riscv/Kconfig | 2 ++ samples/ftrace/ftrace-direct-modify.c | 34 ++++++++++++++++++ samples/ftrace/ftrace-direct-multi-modify.c | 40 +++++++++++++++++++++ samples/ftrace/ftrace-direct-multi.c | 24 +++++++++++++ samples/ftrace/ftrace-direct-too.c | 27 ++++++++++++++ samples/ftrace/ftrace-direct.c | 23 ++++++++++++ 6 files changed, 150 insertions(+)
arch/riscv/Kconfig +2 -0
@@ -122,6 +122,8 @@ config RISCV select HAVE_POSIX_CPU_TIMERS_TASK_WORK select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RSEQ + select HAVE_SAMPLE_FTRACE_DIRECT + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI select HAVE_STACKPROTECTOR select HAVE_SYSCALL_TRACEPOINTS select IRQ_DOMAIN
samples/ftrace/ftrace-direct-modify.c +34 -0
@@ -22,6 +22,40 @@ extern void my_tramp2(void *);static unsigned long my_ip = (unsigned long )schedule; +#ifdef CONFIG_RISCV + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp1, @function\n" +" .globl my_tramp1\n" +" my_tramp1:\n" +" addi sp,sp,-16\n" +" sd t0,0(sp)\n" +" sd ra,8(sp)\n" +" call my_direct_func1\n" +" ld t0,0(sp)\n" +" ld ra,8(sp)\n" +" addi sp,sp,16\n" +" jr t0\n" +" .size my_tramp1, .-my_tramp1\n" + +" .type my_tramp2, @function\n" +" .globl my_tramp2\n" +" my_tramp2:\n" +" addi sp,sp,-16\n" +" sd t0,0(sp)\n" +" sd ra,8(sp)\n" +" call my_direct_func2\n" +" ld t0,0(sp)\n" +" ld ra,8(sp)\n" +" addi sp,sp,16\n" +" jr t0\n" +" .size my_tramp2, .-my_tramp2\n" +" .popsection\n" +); + +#endif /* CONFIG_RISCV */ +#ifdef CONFIG_X86_64 #include <asm/ibt.h>
samples/ftrace/ftrace-direct-multi-modify.c +40 -0
@@ -20,6 +20,46 @@ void my_direct_func2(unsigned long ip)extern void my_tramp1 (void *);extern void my_tramp2 (void *); +#ifdef CONFIG_RISCV + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp1, @function\n" +" .globl my_tramp1\n" +" my_tramp1:\n" +" addi sp,sp,-24\n" +" sd a0,0(sp)\n" +" sd t0,8(sp)\n" +" sd ra,16(sp)\n" +" mv a0,t0\n" +" call my_direct_func1\n" +" ld a0,0(sp)\n" +" ld t0,8(sp)\n" +" ld ra,16(sp)\n" +" addi sp,sp,24\n" +" jr t0\n" +" .size my_tramp1, .-my_tramp1\n" + +" .type my_tramp2, @function\n" +" .globl my_tramp2\n" +" my_tramp2:\n" +" addi sp,sp,-24\n" +" sd a0,0(sp)\n" +" sd t0,8(sp)\n" +" sd ra,16(sp)\n" +" mv a0,t0\n" +" call my_direct_func2\n" +" ld a0,0(sp)\n" +" ld t0,8(sp)\n" +" ld ra,16(sp)\n" +" addi sp,sp,24\n" +" jr t0\n" +" .size my_tramp2, .-my_tramp2\n" +" .popsection\n" +); + +#endif /* CONFIG_RISCV */ +#ifdef CONFIG_X86_64 #include <asm/ibt.h>
samples/ftrace/ftrace-direct-multi.c +24 -0
@@ -15,6 +15,30 @@ void my_direct_func(unsigned long ip)extern void my_tramp (void *); +#ifdef CONFIG_RISCV + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp, @function\n" +" .globl my_tramp\n" +" my_tramp:\n" +" addi sp,sp,-24\n" +" sd a0,0(sp)\n" +" sd t0,8(sp)\n" +" sd ra,16(sp)\n" +" mv a0,t0\n" +" call my_direct_func\n" +" ld a0,0(sp)\n" +" ld t0,8(sp)\n" +" ld ra,16(sp)\n" +" addi sp,sp,24\n" +" jr t0\n" +" .size my_tramp, .-my_tramp\n" +" .popsection\n" +); + +#endif /* CONFIG_RISCV */ +#ifdef CONFIG_X86_64 #include <asm/ibt.h>
samples/ftrace/ftrace-direct-too.c +27 -0
@@ -17,6 +17,33 @@ void my_direct_func(struct vm_area_struct *vma,extern void my_tramp (void *); +#ifdef CONFIG_RISCV + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp, @function\n" +" .globl my_tramp\n" +" my_tramp:\n" +" addi sp,sp,-40\n" +" sd a0,0(sp)\n" +" sd a1,8(sp)\n" +" sd a2,16(sp)\n" +" sd t0,24(sp)\n" +" sd ra,32(sp)\n" +" call my_direct_func\n" +" ld a0,0(sp)\n" +" ld a1,8(sp)\n" +" ld a2,16(sp)\n" +" ld t0,24(sp)\n" +" ld ra,32(sp)\n" +" addi sp,sp,40\n" +" jr t0\n" +" .size my_tramp, .-my_tramp\n" +" .popsection\n" +); + +#endif /* CONFIG_RISCV */ +#ifdef CONFIG_X86_64 #include <asm/ibt.h>
samples/ftrace/ftrace-direct.c +23 -0
@@ -14,6 +14,29 @@ void my_direct_func(struct task_struct *p)extern void my_tramp (void *); +#ifdef CONFIG_RISCV + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp, @function\n" +" .globl my_tramp\n" +" my_tramp:\n" +" addi sp,sp,-24\n" +" sd a0,0(sp)\n" +" sd t0,8(sp)\n" +" sd ra,16(sp)\n" +" call my_direct_func\n" +" ld a0,0(sp)\n" +" ld t0,8(sp)\n" +" ld ra,16(sp)\n" +" addi sp,sp,24\n" +" jr t0\n" +" .size my_tramp, .-my_tramp\n" +" .popsection\n" +); + +#endif /* CONFIG_RISCV */ +#ifdef CONFIG_X86_64 #include <asm/ibt.h>
Signature · 1 line
2.20.1
PATCH v11 5/5 samples: ftrace: Make the riscv samples support RV32I
In reply to Song ShuaiSince the commit f32b4b467ebd ("RISC-V: enable dynamic ftrace for RV32I") enables dynamic ftrace for RV32I, make these riscv samples also support RV32I.
--- samples/ftrace/ftrace-direct-modify.c | 27 +++++++++-------- samples/ftrace/ftrace-direct-multi-modify.c | 33 +++++++++++---------- samples/ftrace/ftrace-direct-multi.c | 17 ++++++----- samples/ftrace/ftrace-direct-too.c | 25 ++++++++-------- samples/ftrace/ftrace-direct.c | 17 ++++++----- 5 files changed, 62 insertions(+), 57 deletions(-)
samples/ftrace/ftrace-direct-modify.c +14 -13
@@ -23,32 +23,33 @@ extern void my_tramp2(void *);static unsigned long my_ip = (unsigned long )schedule;#ifdef CONFIG_RISCV +#include <asm/asm.h> asm (" .pushsection .text, \"ax\", @progbits\n" " .type my_tramp1, @function\n" " .globl my_tramp1\n" " my_tramp1:\n" -" addi sp,sp,-16\n" -" sd t0,0(sp)\n" -" sd ra,8(sp)\n" +" addi sp,sp,-2*" SZREG "\n" +" " REG_S " t0,0*" SZREG "(sp)\n" +" " REG_S " ra,1*" SZREG "(sp)\n" " call my_direct_func1\n" -" ld t0,0(sp)\n" -" ld ra,8(sp)\n" -" addi sp,sp,16\n" +" " REG_L " t0,0*" SZREG "(sp)\n" +" " REG_L " ra,1*" SZREG "(sp)\n" +" addi sp,sp,2*" SZREG "\n" " jr t0\n" " .size my_tramp1, .-my_tramp1\n" -" .type my_tramp2, @function\n" " .globl my_tramp2\n" +" my_tramp2:\n" -" addi sp,sp,-16\n" -" sd t0,0(sp)\n" -" sd ra,8(sp)\n" +" addi sp,sp,-2*" SZREG "\n" +" " REG_S " t0,0*" SZREG "(sp)\n" +" " REG_S " ra,1*" SZREG "(sp)\n" " call my_direct_func2\n" -" ld t0,0(sp)\n" -" ld ra,8(sp)\n" -" addi sp,sp,16\n" +" " REG_L " t0,0*" SZREG "(sp)\n" +" " REG_L " ra,1*" SZREG "(sp)\n" +" addi sp,sp,2*" SZREG "\n" " jr t0\n" " .size my_tramp2, .-my_tramp2\n" " .popsection\n"
samples/ftrace/ftrace-direct-multi-modify.c +17 -16
@@ -21,38 +21,39 @@ extern void my_tramp1(void *);extern void my_tramp2 (void *);#ifdef CONFIG_RISCV +#include <asm/asm.h> asm (" .pushsection .text, \"ax\", @progbits\n" " .type my_tramp1, @function\n" " .globl my_tramp1\n" " my_tramp1:\n" -" addi sp,sp,-24\n" -" sd a0,0(sp)\n" -" sd t0,8(sp)\n" -" sd ra,16(sp)\n" +" addi sp,sp,-3*" SZREG "\n" +" " REG_S " a0,0*" SZREG "(sp)\n" +" " REG_S " t0,1*" SZREG "(sp)\n" +" " REG_S " ra,2*" SZREG "(sp)\n" " mv a0,t0\n" " call my_direct_func1\n" -" ld a0,0(sp)\n" -" ld t0,8(sp)\n" -" ld ra,16(sp)\n" -" addi sp,sp,24\n" +" " REG_L " a0,0*" SZREG "(sp)\n" +" " REG_L " t0,1*" SZREG "(sp)\n" +" " REG_L " ra,2*" SZREG "(sp)\n" +" addi sp,sp,3*" SZREG "\n" " jr t0\n" " .size my_tramp1, .-my_tramp1\n" " .type my_tramp2, @function\n" " .globl my_tramp2\n" " my_tramp2:\n" -" addi sp,sp,-24\n" -" sd a0,0(sp)\n" -" sd t0,8(sp)\n" -" sd ra,16(sp)\n" +" addi sp,sp,-3*" SZREG "\n" +" " REG_S " a0,0*" SZREG "(sp)\n" +" " REG_S " t0,1*" SZREG "(sp)\n" +" " REG_S " ra,2*" SZREG "(sp)\n" " mv a0,t0\n" " call my_direct_func2\n" -" ld a0,0(sp)\n" -" ld t0,8(sp)\n" -" ld ra,16(sp)\n" -" addi sp,sp,24\n" +" " REG_L " a0,0*" SZREG "(sp)\n" +" " REG_L " t0,1*" SZREG "(sp)\n" +" " REG_L " ra,2*" SZREG "(sp)\n" +" addi sp,sp,3*" SZREG "\n" " jr t0\n" " .size my_tramp2, .-my_tramp2\n" " .popsection\n"
samples/ftrace/ftrace-direct-multi.c +9 -8
@@ -16,22 +16,23 @@ void my_direct_func(unsigned long ip)extern void my_tramp (void *);#ifdef CONFIG_RISCV +#include <asm/asm.h> asm (" .pushsection .text, \"ax\", @progbits\n" " .type my_tramp, @function\n" " .globl my_tramp\n" " my_tramp:\n" -" addi sp,sp,-24\n" -" sd a0,0(sp)\n" -" sd t0,8(sp)\n" -" sd ra,16(sp)\n" +" addi sp,sp,-3*" SZREG "\n" +" " REG_S " a0,0*" SZREG "(sp)\n" +" " REG_S " t0,1*" SZREG "(sp)\n" +" " REG_S " ra,2*" SZREG "(sp)\n" " mv a0,t0\n" " call my_direct_func\n" -" ld a0,0(sp)\n" -" ld t0,8(sp)\n" -" ld ra,16(sp)\n" -" addi sp,sp,24\n" +" " REG_L " a0,0*" SZREG "(sp)\n" +" " REG_L " t0,1*" SZREG "(sp)\n" +" " REG_L " ra,2*" SZREG "(sp)\n" +" addi sp,sp,3*" SZREG "\n" " jr t0\n" " .size my_tramp, .-my_tramp\n" " .popsection\n"
samples/ftrace/ftrace-direct-too.c +13 -12
@@ -18,25 +18,26 @@ void my_direct_func(struct vm_area_struct *vma,extern void my_tramp (void *);#ifdef CONFIG_RISCV +#include <asm/asm.h> asm (" .pushsection .text, \"ax\", @progbits\n" " .type my_tramp, @function\n" " .globl my_tramp\n" " my_tramp:\n" -" addi sp,sp,-40\n" -" sd a0,0(sp)\n" -" sd a1,8(sp)\n" -" sd a2,16(sp)\n" -" sd t0,24(sp)\n" -" sd ra,32(sp)\n" +" addi sp,sp,-5*" SZREG "\n" +" " REG_S " a0,0*" SZREG "(sp)\n" +" " REG_S " a1,1*" SZREG "(sp)\n" +" " REG_S " a2,2*" SZREG "(sp)\n" +" " REG_S " t0,3*" SZREG "(sp)\n" +" " REG_S " ra,4*" SZREG "(sp)\n" " call my_direct_func\n" -" ld a0,0(sp)\n" -" ld a1,8(sp)\n" -" ld a2,16(sp)\n" -" ld t0,24(sp)\n" -" ld ra,32(sp)\n" -" addi sp,sp,40\n" +" " REG_L " a0,0*" SZREG "(sp)\n" +" " REG_L " a1,1*" SZREG "(sp)\n" +" " REG_L " a2,2*" SZREG "(sp)\n" +" " REG_L " t0,3*" SZREG "(sp)\n" +" " REG_L " ra,4*" SZREG "(sp)\n" +" addi sp,sp,5*" SZREG "\n" " jr t0\n" " .size my_tramp, .-my_tramp\n" " .popsection\n"
samples/ftrace/ftrace-direct.c +9 -8
@@ -15,21 +15,22 @@ void my_direct_func(struct task_struct *p)extern void my_tramp (void *);#ifdef CONFIG_RISCV +#include <asm/asm.h> asm (" .pushsection .text, \"ax\", @progbits\n" " .type my_tramp, @function\n" " .globl my_tramp\n" " my_tramp:\n" -" addi sp,sp,-24\n" -" sd a0,0(sp)\n" -" sd t0,8(sp)\n" -" sd ra,16(sp)\n" +" addi sp,sp,-3*" SZREG "\n" +" " REG_S " a0,0*" SZREG "(sp)\n" +" " REG_S " t0,1*" SZREG "(sp)\n" +" " REG_S " ra,2*" SZREG "(sp)\n" " call my_direct_func\n" -" ld a0,0(sp)\n" -" ld t0,8(sp)\n" -" ld ra,16(sp)\n" -" addi sp,sp,24\n" +" " REG_L " a0,0*" SZREG "(sp)\n" +" " REG_L " t0,1*" SZREG "(sp)\n" +" " REG_L " ra,2*" SZREG "(sp)\n" +" addi sp,sp,3*" SZREG "\n" " jr t0\n" " .size my_tramp, .-my_tramp\n" " .popsection\n"
Signature · 7 lines
2.20.1 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Ping...
Quoted 142 lines · Song Shuai
Changes in V11:
- append a patch that makes the
DIRECT_CALLsamples support RV32I in this series fixing the rv32 build failure reported by Palmer- validated with ftrace boottime selftest and manual sample modules test in qemu-system for RV32I and RV64I
This series optimizes function trace. The first 3 independent patches has been picked in the V7 version of this series, the subsequent version continues the following 4 patches:
select
FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY[1] (patch 1)
==========================================================In RISC-V, -fpatchable-function-entry option is used to support dynamic ftrace in this commit
afc76b8b8011("riscv: UsingPATCHABLE_FUNCTION_ENTRYinstead of MCOUNT"). So recordmcount don't have to be called to create the__mcount_locsection before the vmlinux linking.Here selects
FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRYto tellMakefilenot to run recordmcount.Make function graph use ftrace directly [2] (patch 2)
========================================================In RISC-V architecture, when we enable the
ftrace_graphtracer on some functions, the function tracings on other functions will suffer extra graph tracing work. In essence,graph_opsisn't limited by itsfunc_hashdue to the global ftrace_graph_[regs]_call label. That should be corrected.What inspires me is the commit
0c0593b45c9b("x86/ftrace: Make function graph use ftrace directly") that usesgraph_ops::funcfunction to installreturn_hookerand makes the function called against itsfunc_hash.Add
WITH_DIRECT_CALLSsupport [3] (patch 3, 4)
==============================================This series adds
DYNAMIC_FTRACE_WITH_DIRECT_CALLSsupport for RISC-V.SAMPLE_FTRACE_DIRECTandSAMPLE_FTRACE_DIRECT_MULTIare also included here as the samples for testingDIRECT_CALLSrelated interface.First, select the
DYNAMIC_FTRACE_WITH_DIRECT_CALLSto provide register_ftrace_direct[_multi] interfaces allowing user to register the customed trampoline (direct_caller) as the mcount for one or more target functions. And modify_ftrace_direct[_multi] are also provided for modifydirect_caller.At the same time, the samples in
./samples/ftrace/can be built as kerenl module for testing these interfaces withSAMPLE_FTRACE_DIRECTandSAMPLE_FTRACE_DIRECT_MULTIselected.Second, to make the
direct_callerand the other ftrace hooks (eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary register
are nominated to store the address ofdirect_callerinftrace_regs_caller.
After the setting of the addressdirect_callerbydirect_ops->funcand theRESTORE_REGSinftrace_regs_caller,direct_callerwill be jumped to by thejrinst.The series's old changes related these patches
==========================================Changes in v10:
https://lore.kernel.org/all/[email protected]/- add Acked-by from Björn Töpel in patch 2 and patch 4
- replacemovewithmvin patch3
- prettify patch 2/4 with proper tabsChanges in v9:
https://lore.kernel.org/linux-riscv/[email protected]/1. add Acked-by from Björn Töpel in patch 1
2. rebase patch2/patch3 on Linux v6.4-rc1
- patch 2: to make the
SAVE_ABI_REGSconfigurable, revert the modification ofmcount-dyn.Sfrom commit (45b32b946a97"riscv:
entry: Consolidate general regs saving/restoring")- patch 3: to pass the
trace_selftest, add the implement offtrace_stub_direct_trampfrom commit (fee86a4ed536"ftrace:
selftest: remove brokentrace_direct_tramp") ; and fixup the context conflict inKconfigChanges in v8:
https://lore.kernel.org/linux-riscv/[email protected]/
- Fix incorrect address values in the 4nd patch
- Rebased on v6.3-rc2Changes in v7:
https://lore.kernel.org/linux-riscv/[email protected]/
- FixupRESTORE_ABI_REGSby removePT_T0(sp)overwrite.
- AddFTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY[1]
- Fixup kconfig withHAVE_SAMPLE_FTRACE_DIRECT&HAVE_SAMPLE_FTRACE_DIRECT_MULTIChanges in v6:
https://lore.kernel.org/linux-riscv/[email protected]/
- Replace 8 withMCOUNT_INSN_SIZE
- Replace "REG_La1,PT_RA(sp)" with "mv a1, ra"
- Add Evgenii Shatokhin commentChanges in v5:
https://lore.kernel.org/linux-riscv/[email protected]/
- SortKconfigentries in alphabetical order.Changes in v4:
https://lore.kernel.org/linux-riscv/[email protected]/
- Include [3] for maintenance. [Song Shuai]Changes in V3:
https://lore.kernel.org/linux-riscv/[email protected]/
- Include [2] for maintenance. [Song Shuai][1]: https://lore.kernel.org/linux-riscv/CAAYs2=j3Eak9vU6xbAw0zPuoh00rh8v5C2U3fePkokZFibWs2g@mail.gmail.com/T/#t
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://lore.kernel.org/linux-riscv/[email protected]/Song Shuai (5):
riscv: selectFTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
riscv: ftrace: Addftrace_graph_func
riscv: ftrace: AddDYNAMIC_FTRACE_WITH_DIRECT_CALLSsupport
samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
samples: ftrace: Make the riscv samples support RV32Iarch/riscv/Kconfig | 4 + arch/riscv/include/asm/ftrace.h | 19 +- arch/riscv/kernel/ftrace.c | 30 ++- arch/riscv/kernel/mcount-dyn.S | 200 ++++++++++++++++---- samples/ftrace/ftrace-direct-modify.c | 35 ++++ samples/ftrace/ftrace-direct-multi-modify.c | 41 ++++ samples/ftrace/ftrace-direct-multi.c | 25 +++ samples/ftrace/ftrace-direct-too.c | 28 +++ samples/ftrace/ftrace-direct.c | 24 +++ 9 files changed, 350 insertions(+), 56 deletions(-)
Signature · 2 lines
Thanks Song Shuai
Quoted 1 line · Song Shuai
Ping...
A context-less ping is not very helpful - what are you looking for here? More reviews? For example, someone to look at 5/5?
Quoted 1 line · Song Shuai
If it's application you want, you sent the patch only last week - which was during the merge window, making it unlikely to be applied.
Either way, please try to explain what it is that you are looking for when you do a ping!
Cheers,
Conor.
Quoted 5 lines · Conor Dooley
Quoted 1 line · Song Shuai
Ping...
A context-less ping is not very helpful - what are you looking for here? More reviews? For example, someone to look at 5/5? >
Sorry for the context-less ping. I hoped someone could look at the 5th patch.
Quoted 6 lines
Quoted 1 line · Song Shuai
If it's application you want, you sent the patch only last week - which was during the merge window, making it unlikely to be applied.
Quoted 1 line
Either way, please try to explain what it is that you are looking for
when you do a ping!
Thanks for your correction, I'll follow this thread after the merge window closes.
Quoted 4 lines
Cheers,
Conor.
Signature · 2 lines
Thanks Song Shuai
Song Shuai <[email protected]> writes:
[...]
Quoted 2 lines
Add
WITH_DIRECT_CALLSsupport [3] (patch 3, 4)
==============================================
We've had some offlist discussions, so here's some input for a wider audience! Most importantly, this is for Palmer, so that this series is not merged until a proper BPF trampoline fix is in place.
Note that what's currently usable from BPF trampoline *works*. It's when this series is added that it breaks.
TL;DR This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS, which enables fentry/fexit BPF trampoline support. Unfortunately the fexit/BPF_TRAMP_F_SKIP_FRAME parts of the RV BPF trampoline breaks with this addition, and need to be addressed *prior* merging this series. An easy way to reproduce, is just calling any of the kselftest tests that uses fexit patching.
The issue is around the nop seld, and how a call is done; The nop sled (patchable-function-entry) size changed from 16B to 8B in commit 6724a76cff85 ("riscv: ftrace: Reduce the detour code size to half"), but BPF code still uses the old 16B. So it'll work for BPF programs, but not for regular kernel functions.
An example:
| ffffffff80fa4150 <bpf_fentry_test1>: | ffffffff80fa4150: 0001 nop | ffffffff80fa4152: 0001 nop | ffffffff80fa4154: 0001 nop | ffffffff80fa4156: 0001 nop | ffffffff80fa4158: 1141 add sp,sp,-16 | ffffffff80fa415a: e422 sd s0,8(sp) | ffffffff80fa415c: 0800 add s0,sp,16 | ffffffff80fa415e: 6422 ld s0,8(sp) | ffffffff80fa4160: 2505 addw a0,a0,1 | ffffffff80fa4162: 0141 add sp,sp,16 | ffffffff80fa4164: 8082 ret
is patched to:
| ffffffff80fa4150: f70c0297 auipc t0,-150208512 | ffffffff80fa4154: eb0282e7 jalr t0,t0,-336
The return address to bpf_fentry_test1 is stored in t0 at BPF trampoline entry. Return to the *parent* is in ra. The trampline has to deal with this.
For BPF_TRAMP_F_SKIP_FRAME/CALL_ORIG, the BPF trampoline will skip too many bytes, and not correctly handle parent calls.
Further; The BPF trampoline currently has a different way of patching the nops for BPF programs, than what ftrace does. That should be changed to match what ftrace does (auipc/jalr t0).
To summarize:
* Align BPF nop sled with patchable-function-entry: 8B.
* Adapt BPF trampoline for 8B nop sleds.
* Adapt BPF trampoline t0 return, ra parent scheme.
Cheers,
Björn
Quoted 54 lines · On Wed, 12 Jul 2023 11:11:08 PDT (-0700), [email protected] wrote:
Song Shuai <[email protected]> writes:
[...]
Quoted 2 lines
Add
WITH_DIRECT_CALLSsupport [3] (patch 3, 4)
==============================================We've had some offlist discussions, so here's some input for a wider audience! Most importantly, this is for Palmer, so that this series is not merged until a proper BPF trampoline fix is in place.
Note that what's currently usable from BPF trampoline *works*. It's when this series is added that it breaks.
TL;DR This series adds
DYNAMIC_FTRACE_WITH_DIRECT_CALLS, which enables fentry/fexit BPF trampoline support. Unfortunately the fexit/BPF_TRAMP_F_SKIP_FRAME parts of the RV BPF trampoline breaks with this addition, and need to be addressed *prior* merging this series. An easy way to reproduce, is just calling any of the kselftest tests that uses fexit patching.The issue is around the nop seld, and how a call is done; The nop sled (patchable-function-entry) size changed from 16B to 8B in commit
6724a76cff85("riscv: ftrace: Reduce the detour code size to half"), but BPF code still uses the old 16B. So it'll work for BPF programs, but not for regular kernel functions.An example:
| ffffffff80fa4150 <bpf_fentry_test1>: | ffffffff80fa4150: 0001 nop | ffffffff80fa4152: 0001 nop | ffffffff80fa4154: 0001 nop | ffffffff80fa4156: 0001 nop | ffffffff80fa4158: 1141 add sp,sp,-16 | ffffffff80fa415a: e422 sd s0,8(sp) | ffffffff80fa415c: 0800 add s0,sp,16 | ffffffff80fa415e: 6422 ld s0,8(sp) | ffffffff80fa4160: 2505 addw a0,a0,1 | ffffffff80fa4162: 0141 add sp,sp,16 | ffffffff80fa4164: 8082 retis patched to:
| ffffffff80fa4150: f70c0297 auipc t0,-150208512 | ffffffff80fa4154: eb0282e7 jalr t0,t0,-336The return address to
bpf_fentry_test1is stored in t0 at BPF trampoline entry. Return to the *parent* is in ra. The trampline has to deal with this.For BPF_TRAMP_F_SKIP_FRAME/CALL_ORIG, the BPF trampoline will skip too many bytes, and not correctly handle parent calls.
Quoted 8 lines
Further; The BPF trampoline currently has a different way of patching the nops for BPF programs, than what ftrace does. That should be changed to match what ftrace does (auipc/jalr t0).
To summarize:
* Align BPF nop sled with patchable-function-entry: 8B.
* Adapt BPF trampoline for 8B nop sleds.
* Adapt BPF trampoline t0 return, ra parent scheme.
Thanks for digging into this one, I agree we need to sort out the BPF breakages before we merge this. Sounds like there's a rabbit hole here, but hopefully we can get it sorted out.
I've dropped this from patchwork and such, as we'll need at least another spin.
Quoted 2 lines
Cheers,
Björn
Quoted 64 lines · Palmer Dabbelt
Quoted 54 lines · On Wed, 12 Jul 2023 11:11:08 PDT (-0700), [email protected] wrote:
Song Shuai <[email protected]> writes:
[...]
Quoted 2 lines
Add
WITH_DIRECT_CALLSsupport [3] (patch 3, 4)
==============================================We've had some offlist discussions, so here's some input for a wider audience! Most importantly, this is for Palmer, so that this series is not merged until a proper BPF trampoline fix is in place.
Note that what's currently usable from BPF trampoline *works*. It's when this series is added that it breaks.
TL;DR This series adds
DYNAMIC_FTRACE_WITH_DIRECT_CALLS, which enables fentry/fexit BPF trampoline support. Unfortunately the fexit/BPF_TRAMP_F_SKIP_FRAME parts of the RV BPF trampoline breaks with this addition, and need to be addressed *prior* merging this series. An easy way to reproduce, is just calling any of the kselftest tests that uses fexit patching.The issue is around the nop seld, and how a call is done; The nop sled (patchable-function-entry) size changed from 16B to 8B in commit
6724a76cff85("riscv: ftrace: Reduce the detour code size to half"), but BPF code still uses the old 16B. So it'll work for BPF programs, but not for regular kernel functions.An example:
| ffffffff80fa4150 <bpf_fentry_test1>: | ffffffff80fa4150: 0001 nop | ffffffff80fa4152: 0001 nop | ffffffff80fa4154: 0001 nop | ffffffff80fa4156: 0001 nop | ffffffff80fa4158: 1141 add sp,sp,-16 | ffffffff80fa415a: e422 sd s0,8(sp) | ffffffff80fa415c: 0800 add s0,sp,16 | ffffffff80fa415e: 6422 ld s0,8(sp) | ffffffff80fa4160: 2505 addw a0,a0,1 | ffffffff80fa4162: 0141 add sp,sp,16 | ffffffff80fa4164: 8082 retis patched to:
| ffffffff80fa4150: f70c0297 auipc t0,-150208512 | ffffffff80fa4154: eb0282e7 jalr t0,t0,-336The return address to
bpf_fentry_test1is stored in t0 at BPF trampoline entry. Return to the *parent* is in ra. The trampline has to deal with this.For BPF_TRAMP_F_SKIP_FRAME/CALL_ORIG, the BPF trampoline will skip too many bytes, and not correctly handle parent calls.
Quoted 8 lines
Further; The BPF trampoline currently has a different way of patching the nops for BPF programs, than what ftrace does. That should be changed to match what ftrace does (auipc/jalr t0).
To summarize:
* Align BPF nop sled with patchable-function-entry: 8B.
* Adapt BPF trampoline for 8B nop sleds.
* Adapt BPF trampoline t0 return, ra parent scheme.
Quoted 6 lines
Thanks for digging into this one, I agree we need to sort out the BPF breakages before we merge this. Sounds like there's a rabbit hole here, but hopefully we can get it sorted out.
I've dropped this from patchwork and such, as we'll need at least another spin.
Palmer,
The needed BPF patch is upstream in the bpf-next tree, and has been for a couple of weeks.
I think this series is a candidate for RISC-V -next! It would help RISC-V BPF a lot in terms of completeness.
Björn
Quoted 73 lines · Björn Töpel
Quoted 64 lines · Palmer Dabbelt
Quoted 54 lines · On Wed, 12 Jul 2023 11:11:08 PDT (-0700), [email protected] wrote:
Song Shuai <[email protected]> writes:
[...]
Quoted 2 lines
Add
WITH_DIRECT_CALLSsupport [3] (patch 3, 4)
==============================================We've had some offlist discussions, so here's some input for a wider audience! Most importantly, this is for Palmer, so that this series is not merged until a proper BPF trampoline fix is in place.
Note that what's currently usable from BPF trampoline *works*. It's when this series is added that it breaks.
TL;DR This series adds
DYNAMIC_FTRACE_WITH_DIRECT_CALLS, which enables fentry/fexit BPF trampoline support. Unfortunately the fexit/BPF_TRAMP_F_SKIP_FRAME parts of the RV BPF trampoline breaks with this addition, and need to be addressed *prior* merging this series. An easy way to reproduce, is just calling any of the kselftest tests that uses fexit patching.The issue is around the nop seld, and how a call is done; The nop sled (patchable-function-entry) size changed from 16B to 8B in commit
6724a76cff85("riscv: ftrace: Reduce the detour code size to half"), but BPF code still uses the old 16B. So it'll work for BPF programs, but not for regular kernel functions.An example:
| ffffffff80fa4150 <bpf_fentry_test1>: | ffffffff80fa4150: 0001 nop | ffffffff80fa4152: 0001 nop | ffffffff80fa4154: 0001 nop | ffffffff80fa4156: 0001 nop | ffffffff80fa4158: 1141 add sp,sp,-16 | ffffffff80fa415a: e422 sd s0,8(sp) | ffffffff80fa415c: 0800 add s0,sp,16 | ffffffff80fa415e: 6422 ld s0,8(sp) | ffffffff80fa4160: 2505 addw a0,a0,1 | ffffffff80fa4162: 0141 add sp,sp,16 | ffffffff80fa4164: 8082 retis patched to:
| ffffffff80fa4150: f70c0297 auipc t0,-150208512 | ffffffff80fa4154: eb0282e7 jalr t0,t0,-336The return address to
bpf_fentry_test1is stored in t0 at BPF trampoline entry. Return to the *parent* is in ra. The trampline has to deal with this.For BPF_TRAMP_F_SKIP_FRAME/CALL_ORIG, the BPF trampoline will skip too many bytes, and not correctly handle parent calls.
Quoted 8 lines
Further; The BPF trampoline currently has a different way of patching the nops for BPF programs, than what ftrace does. That should be changed to match what ftrace does (auipc/jalr t0).
To summarize:
* Align BPF nop sled with patchable-function-entry: 8B.
* Adapt BPF trampoline for 8B nop sleds.
* Adapt BPF trampoline t0 return, ra parent scheme.Quoted 6 lines
Thanks for digging into this one, I agree we need to sort out the BPF breakages before we merge this. Sounds like there's a rabbit hole here, but hopefully we can get it sorted out.
I've dropped this from patchwork and such, as we'll need at least another spin.
Quoted 7 lines
Palmer,
The needed BPF patch is upstream in the bpf-next tree, and has been for a couple of weeks.
I think this series is a candidate for RISC-V -next! It would help RISC-V BPF a lot in terms of completeness.
Palmer,
The needed fix for BPF is now in Linus' tree, commit 25ad10658dc1 ("riscv, bpf: Adapt bpf trampoline to optimized riscv ftrace framework"). IOW, this ftrace series can be merged now.
Björn
Quoted 54 lines · Björn Töpel
Song Shuai <[email protected]> writes:
[...]
Quoted 2 lines
Add
WITH_DIRECT_CALLSsupport [3] (patch 3, 4)
==============================================We've had some offlist discussions, so here's some input for a wider audience! Most importantly, this is for Palmer, so that this series is not merged until a proper BPF trampoline fix is in place.
Note that what's currently usable from BPF trampoline *works*. It's when this series is added that it breaks.
TL;DR This series adds
DYNAMIC_FTRACE_WITH_DIRECT_CALLS, which enables fentry/fexit BPF trampoline support. Unfortunately the fexit/BPF_TRAMP_F_SKIP_FRAME parts of the RV BPF trampoline breaks with this addition, and need to be addressed *prior* merging this series. An easy way to reproduce, is just calling any of the kselftest tests that uses fexit patching.The issue is around the nop seld, and how a call is done; The nop sled (patchable-function-entry) size changed from 16B to 8B in commit
6724a76cff85("riscv: ftrace: Reduce the detour code size to half"), but BPF code still uses the old 16B. So it'll work for BPF programs, but not for regular kernel functions.An example:
| ffffffff80fa4150 <bpf_fentry_test1>: | ffffffff80fa4150: 0001 nop | ffffffff80fa4152: 0001 nop | ffffffff80fa4154: 0001 nop | ffffffff80fa4156: 0001 nop | ffffffff80fa4158: 1141 add sp,sp,-16 | ffffffff80fa415a: e422 sd s0,8(sp) | ffffffff80fa415c: 0800 add s0,sp,16 | ffffffff80fa415e: 6422 ld s0,8(sp) | ffffffff80fa4160: 2505 addw a0,a0,1 | ffffffff80fa4162: 0141 add sp,sp,16 | ffffffff80fa4164: 8082 retis patched to:
| ffffffff80fa4150: f70c0297 auipc t0,-150208512 | ffffffff80fa4154: eb0282e7 jalr t0,t0,-336The return address to
bpf_fentry_test1is stored in t0 at BPF trampoline entry. Return to the *parent* is in ra. The trampline has to deal with this.For BPF_TRAMP_F_SKIP_FRAME/CALL_ORIG, the BPF trampoline will skip too many bytes, and not correctly handle parent calls.
Quoted 9 lines
Further; The BPF trampoline currently has a different way of patching the nops for BPF programs, than what ftrace does. That should be changed to match what ftrace does (auipc/jalr t0).
To summarize:
* Align BPF nop sled with patchable-function-entry: 8B.
* Adapt BPF trampoline for 8B nop sleds.
* Adapt BPF trampoline t0 return, ra parent scheme.
Thanks Björn, I make a adaptation as follows, looking forward to your review.
https://lore.kernel.org/bpf/[email protected]/
Quoted 5 lines
Cheers,
Björn