ftrace.txt: fix typo in function grapher name
Subsystems (3)
PATCH 1/2 ftrace.txt: fix typo in function grapher name
The function graph tracer is called just "function_graph" (no trailing "_tracer" needed).
--- Documentation/trace/ftrace.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
Documentation/trace/ftrace.txt +1 -1
@@ -179,7 +179,7 @@ Here is the list of current tracers that may be configured. Function call tracer to trace all kernel functions. - "function_graph_tracer" + "function_graph" Similar to the function tracer except that the function tracer probes the functions on their entry
Signature · 1 line
1.6.3.1
PATCH 2/2 ftrace: document basic ftracer/ftracer graph needs
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, document the requirements.
--- Documentation/trace/ftrace.txt | 7 ++++++ kernel/trace/Kconfig | 43 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 0 deletions(-)
Documentation/trace/ftrace.txt +7 -0
@@ -27,6 +27,13 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See kernel/trace/Kconfig for details about different options and +the required architecture pieces. + + The File System ---------------
kernel/trace/Kconfig +43 -0
63 lines not on this page · raw message
Signature · 1 line
1.6.3.1
Steven Rostedt
maintainerRe: [PATCH 2/2] ftrace: document basic ftracer/ftracer graph needs
Quoted 67 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, document the requirements.
Signed-off-by Mike Frysinger <[email protected]>--- Documentation/trace/ftrace.txt | 7 ++++++ kernel/trace/Kconfig | 43 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 0 deletions(-)Documentation/trace/ftrace.txt +7
-0@@ -27,6 +27,13 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See kernel/trace/Kconfig for details about different options and +the required architecture pieces. + + The File System ---------------kernel/trace/Kconfig +27
-0@@ -14,9 +14,46 @@ config HAVE_FTRACE_NMI_ENTER config HAVE_FUNCTION_TRACER bool + help + Implement support for the _mcount() function (or whatever your + toolchain names it), and the ftrace_stub function (which only + does a return). + + The _mcount() function should check the function pointer + ftrace_trace_function to see if it is set to ftrace_stub. If + it is, there is nothing for you to do -- return. If it isn't, + then call it the same way the _mcount() function normally calls + __mcount_internal() -- the first argument is the "frompc" while + the second argument is the "selfpc" (adjusted to remove the size + of the embedded _mcount() call). + + For example, if the function foo() calls bar(), when the bar() + function calls _mcount(), the arguments to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) config HAVE_FUNCTION_GRAPH_TRACER bool + help + Update your _mcount() function to check the function pointers + ftrace_graph_return (compare to ftrace_stub) and ftrace_graph_entry + (compare to ftrace_graph_entry_stub). If either of those are not + set to the relevant stub function, call the arch-specific function + ftrace_graph_caller which in turn calls the arch-specific function + prepare_ftrace_return. + + The arguments to prepare_ftrace_return() are slightly different. + The first argument will instead be a pointer to the "frompc" --
kernel/trace/Kconfig config HAVE_FTRACE_NMI_ENTER · line 56
Quoted 10 lines
kernel/trace/Kconfig +10
-0+ typically located in the stack. This allows the function to + hijack the return address temporarily to have it point to the + arch-specific function return_to_handler(). For exact details of + how to implement prepare_ftrace_return(), consult an existing port. + It is fairly straight forward C code. + + The return_to_handler() function is simple as well. Have it call + the common ftrace_return_to_handler() function. The return value + is the original return address, so restore this to the right place + and return.
This does not belong in the Kconfig. That's the last place a developer will look for implementing something. I would recommend adding a
Documentation/trace/ftrace-function-implementation.txt
file.
-- Steve
Quoted 25 lines
kernel/trace/Kconfig +6
-0config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool@@ -25,11 +62,17 @@ config HAVE_FUNCTION_TRACE_MCOUNT_TEST variable at the mcount call site. Otherwise, this variable is tested by the called function. + It is of course optional for arches, but does improve performance + a bit as it prevents constant state saving and call overhead when + the function tracer is actually disabled. + config HAVE_DYNAMIC_FTRACE bool config HAVE_FTRACE_MCOUNT_RECORD bool + help + See scripts/recordmcount.pl for more info. config HAVE_HW_BRANCH_TRACER boolSignature · 1 line
1.6.3.1
Re: [PATCH 2/2] ftrace: document basic ftracer/ftracer graph needs
Quoted 8 lines · Steven Rostedt
Quoted 4 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, document the requirements.
This does not belong in the
Kconfig. That's the last place a developer will look for implementing something.
yes and no -- i was following the existing documenting practice of ftrace where details were added to the Kconfig. guess i should fix that in the process.
there should be at least tips in each HAVE_XXX as to where to look for implementation details.
-mike
Steven Rostedt
maintainerRe: [PATCH 2/2] ftrace: document basic ftracer/ftracer graph needs
Quoted 7 lines · Mike Frysinger
Quoted 6 lines · Steven Rostedt
Quoted 4 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, document the requirements.
Quoted 6 lines
Quoted 2 lines
This does not belong in the
Kconfig. That's the last place a developer will look for implementing something.yes and no -- i was following the existing documenting practice of ftrace where details were added to the
Kconfig. guess i should fix that in the process.
Documenting what should be selected and why is one thing, but to document implementation is another. I'm not against documentation in Kconfig, but that should explain more about why things are selected. But not implementation details.
Quoted 3 lines
there should be at least tips in each
HAVE_XXXas to where to look for implementation details.
That can be opening a can of worms.
Really, the focus of the documentation in Kconfig is about why things must be selected, not how to implement the code to add that selection.
If there's a HAVE_XXX in Kconfig that I want to see how to implement it, I go off and do 'grep -r XXX Documentation'.
-- Steve
Re: [PATCH 2/2] ftrace: document basic ftracer/ftracer graph needs
Quoted 5 lines · Steven Rostedt
Quoted 2 lines · Mike Frysinger
there should be at least tips in each
HAVE_XXXas to where to look for implementation details.That can be opening a can of worms.
no it isnt:
config HAVE_FUNCTION_TRACER
bool
help
Please see Documentation/trace/ftrace-implementation.txt
Quoted 2 lines
Really, the focus of the documentation in
Kconfigis about why things must be selected, not how to implement the code to add that selection.
except that some of these defines come from the exact implementation details.
Quoted 2 lines
If there's a
HAVE_XXXinKconfigthat I want to see how to implement it, I go off and do 'grep -r XXX Documentation'.
i look for the source of the define because ive given up on the Documentation actually being relevant
-mike
Steven Rostedt
maintainerRe: [PATCH 2/2] ftrace: document basic ftracer/ftracer graph needs
Quoted 5 lines · Mike Frysinger
Quoted 4 lines · Steven Rostedt
Quoted 2 lines · Mike Frysinger
there should be at least tips in each
HAVE_XXXas to where to look for implementation details.
Quoted 7 lines
Quoted 1 line
That can be opening a can of worms.
no it isnt:
configHAVE_FUNCTION_TRACER
bool
help
Please seeDocumentation/trace/ftrace-implementation.txt
I'm fine with that, I was thing something else.
Quoted 11 lines
Quoted 3 lines
Really, the focus of the documentation in
Kconfigis about why things must be selected, not how to implement the code to add that selection.except that some of these defines come from the exact implementation details.
Quoted 2 lines
If there's a
HAVE_XXXinKconfigthat I want to see how to implement it, I go off and do 'grep -r XXX Documentation'.i look for the source of the define because ive given up on the Documentation actually being relevant
And you think documentation in Kconfig will be updated??
-- Steve
Re: [PATCH 2/2] ftrace: document basic ftracer/ftracer graph needs
Quoted 9 lines · Steven Rostedt
Quoted 6 lines · Mike Frysinger
Quoted 2 lines · Steven Rostedt
If there's a
HAVE_XXXinKconfigthat I want to see how to implement it, I go off and do 'grep -r XXX Documentation'.i look for the source of the define because ive given up on the Documentation actually being relevant
And you think documentation in
Kconfigwill be updated??
i didnt mean *just* Kconfig, but funny enough, the Kconfig for HAVE_FUNCTION_TRACE_MCOUNT_TEST had the most relevant information. all other places were #ifdef with no meaning. and the existing x86/ppc/sh ports of ftrace are devoid of any comments let alone useful ones, as is the core ftrace code wrt the arch pieces it relies on. the absolute dearth of ftrace documentation for arch porters is ridiculous.
-mike
Steven Rostedt
maintainerRe: [PATCH 2/2] ftrace: document basic ftracer/ftracer graph needs
Quoted 9 lines · Mike Frysinger
Quoted 8 lines · Steven Rostedt
Quoted 6 lines · Mike Frysinger
Quoted 2 lines · Steven Rostedt
If there's a
HAVE_XXXinKconfigthat I want to see how to implement it, I go off and do 'grep -r XXX Documentation'.i look for the source of the define because ive given up on the Documentation actually being relevant
Quoted 6 lines
Quoted 1 line
And you think documentation in
Kconfigwill be updated??i didnt mean *just*
Kconfig, but funny enough, theKconfigforHAVE_FUNCTION_TRACE_MCOUNT_TESThad the most relevant information. all other places were #ifdef with no meaning. and the existing x86/ppc/sh ports of ftrace are devoid of any comments let alone useful
Well that example was commented in the generic code:
kernel/trace/ftrace.c:
#ifndef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
/*
* For those archs that do not test ftrace_trace_stop in their
* mcount call site, we need to do it from C.
*/
And only x86 has it defined, thus looking at the mocunt call site as descibed in the comment, or searching the kernel for ftrace_trace_stop:
arch/x86/kernel/entry_64.S
ENTRY(ftrace_caller)
cmpl $0, function_trace_stop
jne ftrace_stub
But yes, as documenting goes, this is a poor example. The function_trace_stop is more of a way to quickly stop ftrace, in case of emergency. It can also be useful for static ftrace. When ftrace is enabled, there's really not much difference in overhead if it is implemented or not. Only for static tracing, or when ftrace is disabled via the variable (not with dynamic ftrace).
Quoted 3 lines
ones, as is the core ftrace code wrt the arch pieces it relies on. the absolute dearth of ftrace documentation for arch porters is ridiculous.
Yes, we can always document more. But you are the first to complain to me about lack of documentation for porting. I get complaints all the time about lack of documentation for using it, and I'm working on that.
When I get time, I would love to write up a "how to port ftrace" doc.
I would also love to review one ;-)
-- Steve
Re: [PATCH 2/2] ftrace: document basic ftracer/ftracer graph needs
Quoted 7 lines · Steven Rostedt
Quoted 3 lines
ones, as is the core ftrace code wrt the arch pieces it relies on. the absolute dearth of ftrace documentation for arch porters is ridiculous.
Yes, we can always document more. But you are the first to complain to me about lack of documentation for porting. I get complaints all the time about lack of documentation for using it, and I'm working on that.
imo, it should be a given that when a framework or common feature is added which requires some kind of arch support, what exactly is required of the arch should be clearly spelled out. having an arch or two implement said feature is useful sometimes as an example, but it is rarely (if ever) a good place to find out details. more time is spent basically reverse engineering the (unfamiliar) arch details.
this isnt a complaint specific to ftrace btw, so it's more of a general hair pulling mini rant.
Quoted 3 lines
When I get time, I would love to write up a "how to port ftrace" doc.
I would also love to review one ;-)
well ive already done the footwork for the basic options, so i'm starting the documentation now by moving the Kconfig stuff i posted there. but it'll be missing info on:HAVE_FTRACE_NMI_ENTERHAVE_FTRACE_SYSCALLSHAVE_FTRACE_MCOUNT_RECORDHAVE_DYNAMIC_FTRACE
personally i dont care about NMI as the Blackfin arch's handling of NMI is pretty much useless to Linux, but i would be interested in the other three
-mike
Steven Rostedt
maintainerRe: [PATCH 2/2] ftrace: document basic ftracer/ftracer graph needs
In reply to Mike FrysingerQuoted 18 lines · Mike Frysinger
The File System
---------------kernel/trace/Kconfig +7
-0@@ -14,9 +14,46 @@ config HAVE_FTRACE_NMI_ENTER config HAVE_FUNCTION_TRACER bool + help + Implement support for the _mcount() function (or whatever your + toolchain names it), and the ftrace_stub function (which only + does a return). + + The _mcount() function should check the function pointer + ftrace_trace_function to see if it is set to ftrace_stub. If
kernel/trace/Kconfig config HAVE_FTRACE_NMI_ENTER · line 33
Quoted 10 lines
kernel/trace/Kconfig +10
-0+ it is, there is nothing for you to do -- return. If it isn't, + then call it the same way the _mcount() function normally calls + __mcount_internal() -- the first argument is the "frompc" while + the second argument is the "selfpc" (adjusted to remove the size + of the embedded _mcount() call). + + For example, if the function foo() calls bar(), when the bar() + function calls _mcount(), the arguments to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment)
This is too much for Kconfig.
Quoted 13 lines
kernel/trace/Kconfig +10
-0config HAVE_FUNCTION_GRAPH_TRACER bool + help + Update your _mcount() function to check the function pointers + ftrace_graph_return (compare to ftrace_stub) and ftrace_graph_entry + (compare to ftrace_graph_entry_stub). If either of those are not + set to the relevant stub function, call the arch-specific function + ftrace_graph_caller which in turn calls the arch-specific function + prepare_ftrace_return. + + The arguments to prepare_ftrace_return() are slightly different. + The first argument will instead be a pointer to the "frompc" --
kernel/trace/Kconfig config HAVE_FTRACE_NMI_ENTER · line 56
Quoted 10 lines
kernel/trace/Kconfig +10
-0+ typically located in the stack. This allows the function to + hijack the return address temporarily to have it point to the + arch-specific function return_to_handler(). For exact details of + how to implement prepare_ftrace_return(), consult an existing port. + It is fairly straight forward C code. + + The return_to_handler() function is simple as well. Have it call + the common ftrace_return_to_handler() function. The return value + is the original return address, so restore this to the right place + and return.
This is also too much for Kconfig.
kernel/trace/Kconfig config HAVE_FUNCTION_TRACE_MCOUNT_TEST · line 68
Quoted 11 lines
kernel/trace/Kconfig +4
-0config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool@@ -25,11 +62,17 @@ config HAVE_FUNCTION_TRACE_MCOUNT_TEST variable at the mcount call site. Otherwise, this variable is tested by the called function. + It is of course optional for arches, but does improve performance + a bit as it prevents constant state saving and call overhead when + the function tracer is actually disabled. +
The above would be fine, although I would reword it a little:
This is optional for archs, but will improve performance, as it prevents constant state saving and call overhead when the function tracer tracer is active, but has been disabled.
I put in 'is active' because with dynamic function tracing it has no more overhead when disabled. We can enable function tracing, but soft disable it with a check of a bit.
kernel/trace/Kconfig config HAVE_FUNCTION_TRACE_MCOUNT_TEST · line 75
Quoted 7 lines
kernel/trace/Kconfig +2
-0config HAVE_DYNAMIC_FTRACE bool config HAVE_FTRACE_MCOUNT_RECORD bool + help + See scripts/recordmcount.pl for more info.
This is fine.
-- Steve
Quoted 7 lines
kernel/trace/Kconfig +0
-0config HAVE_HW_BRANCH_TRACER boolSignature · 1 line
1.6.3.1
PATCH v2 2/2 ftrace: document basic ftracer/ftracer graph needs
In reply to Mike FrysingerWhile implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
---
v2
- move to Documentation/ and add a lot more details
note: yes, i think every kconfig option that is documented should point to the relevant text file as this helps in the case of people who use kconfig's search function to look for things
Documentation/trace/ftrace-implementation.txt | 223 +++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 242 insertions(+), 3 deletions(-) create mode 100644 Documentation/trace/ftrace-implementation.txt
Documentation/trace/ftrace-implementation.txt new file +223 -0
@@ -0,0 +1,223 @@ + ftrace guts + =========== + +Introduction +------------ + +Here we will cover the architecture pieces that the common ftrace code relies +on for proper functioning. Things are broken down into increasing complexity +so that you can start at the top and get at least basic functionality. + +Note that this focuses on architecture implementation details only. If you +want more explanation of a feature in terms of common code, review the common +ftrace.txt file. + + +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'm(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage/clobbering is a major +problem here. You might also want to look at how glibc has implemented the +mcount function for your architecture. It might be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; + +do_trace: + + /* save all state needed by the ABI */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +} + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address. + +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif + +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Here you are returning from a function, so you might be able to skimp +on things saved/restored. +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point(); +} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- + +If you can't trace NMI functions, then skip this option. + +<details to be filled> + + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled> + + +HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled> + + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>
Documentation/trace/ftrace.txt +6 -0
@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such. + + The File System ---------------
kernel/trace/Kconfig +13 -3
@@ -11,31 +11,41 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_FTRACE_SYSCALLS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE bool
Signature · 1 line
1.6.3.1
Re: [PATCH 2/2 v2] ftrace: document basic ftracer/ftracer graph needs
Quoted 4 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
have you merged this ? i have a small update to it (remind people to export the mcount symbol), but dont know if you'd like a fresh patch or one on top of this.
-mike
Re: [PATCH 2/2 v2] ftrace: document basic ftracer/ftracer graph needs
Quoted 10 lines · Mike Frysinger
Quoted 4 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
have you merged this ? i have a small update to it (remind people to export the mcount symbol), but dont know if you'd like a fresh patch or one on top of this.
-mike
Hi,
It hasn't been merged, you can safely send a v3.
Thanks.
PATCH v3 1/1 ftrace: document basic ftracer/ftracer graph needs
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
---
v3
- add a few more notes about mcount and exporting it
Documentation/trace/ftrace-implementation.txt | 229 +++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 248 insertions(+), 3 deletions(-) create mode 100644 Documentation/trace/ftrace-implementation.txt
Documentation/trace/ftrace-implementation.txt new file +229 -0
@@ -0,0 +1,229 @@ + ftrace guts + =========== + +Introduction +------------ + +Here we will cover the architecture pieces that the common ftrace code relies +on for proper functioning. Things are broken down into increasing complexity +so that you can start at the top and get at least basic functionality. + +Note that this focuses on architecture implementation details only. If you +want more explanation of a feature in terms of common code, review the common +ftrace.txt file. + + +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'main(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount +We'll make the assumption below that the symbol is "mcount" just to keep things +nice and simple in the examples. + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage/clobbering is a major +problem here. You might also want to look at how glibc has implemented the +mcount function for your architecture. It might be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; + +do_trace: + + /* save all state needed by the ABI */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +} + +Don't forget to export mcount for modules ! +extern void mcount(void); +EXPORT_SYMBOL(mcount); + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address. + +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif + +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Here you are returning from a function, so you might be able to skimp +on things saved/restored. +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point(); +} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- + +If you can't trace NMI functions, then skip this option. + +<details to be filled> + + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled> + + +HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled> + + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>
Documentation/trace/ftrace.txt +6 -0
@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such. + + The File System ---------------
kernel/trace/Kconfig +13 -3
@@ -11,31 +11,41 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_FTRACE_SYSCALLS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE bool
Signature · 1 line
1.6.3.1
Re: [PATCH v3] ftrace: document basic ftracer/ftracer graph needs
Quoted 36 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
Signed-off-by Mike Frysinger <[email protected]>---
v3
- add a few more notes about mcount and exporting itDocumentation/trace/ftrace-implementation.txt | 229 +++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 248 insertions(+), 3 deletions(-) create mode 100644 Documentation/trace/ftrace-implementation.txtDocumentation/trace/ftrace-implementation.txt new file +14
-0@@ -0,0 +1,229 @@ + ftrace guts + =========== + +Introduction +------------ + +Here we will cover the architecture pieces that the common ftrace code relies +on for proper functioning. Things are broken down into increasing complexity +so that you can start at the top and get at least basic functionality. + +Note that this focuses on architecture implementation details only. If you +want more explanation of a feature in terms of common code, review the common +ftrace.txt file. +
Documentation/trace/ftrace-implementation.txt line 24
Quoted 10 lines
Documentation/trace/ftrace-implementation.txt new file +10
-0+ +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h + + +HAVE_FUNCTION_GRAPH_TRACER
Should be HAVE_FUNCTION_TRACER, right?
Quoted 58 lines
Documentation/trace/ftrace-implementation.txt new file +58
-0+-------------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'main(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount +We'll make the assumption below that the symbol is "mcount" just to keep things +nice and simple in the examples. + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage/clobbering is a major +problem here. You might also want to look at how glibc has implemented the +mcount function for your architecture. It might be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; +
Documentation/trace/ftrace-implementation.txt line 92
Quoted 10 lines
Documentation/trace/ftrace-implementation.txt new file +10
-0+do_trace: + + /* save all state needed by the ABI */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +}
Nice idea this pseudo code.
There is one crucial missing thing, I mean "save all state needed by the ABI" can be more detailed. do_trace _must_
save the scratch and argument registers to the stack because
the traced function may have parameters passed by registers, initialized things on scratch registers, and this state
must be left intact before calling ftrace_trace_function()
Quoted 24 lines
Documentation/trace/ftrace-implementation.txt new file +24
-0+Don't forget to export mcount for modules ! +extern void mcount(void); +EXPORT_SYMBOL(mcount); + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; +
Documentation/trace/ftrace-implementation.txt line 128
Quoted 10 lines
Documentation/trace/ftrace-implementation.txt new file +10
-0+ extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address.
Indeed, and btw I'm currently checking entry_32.S from x86:
cmpl $ftrace_stub, ftrace_graph_return
jnz ftrace_graph_caller
cmpl $ftrace_graph_entry_stub, ftrace_graph_entry
jnz ftrace_graph_caller
It means:
if (ftrace_graph_return == ftrace_stub)
goto ftrace_graph_caller
else if (ftrace_graph_entry == ftrace_graph_entry_stub)
goto ftrace_graph_caller
This makes no sense. We want to ensure that both the entry and return handler have an appropriate callback. I was confused when I wrote that.
It's harmless but creates pointless checks and then overhead, I'll have to fix it (it's on !DYNAMIC_FTRACE).
Quoted 40 lines
Documentation/trace/ftrace-implementation.txt new file +40
-0+ +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void)
Documentation/trace/ftrace-implementation.txt line 178
Quoted 10 lines
Documentation/trace/ftrace-implementation.txt new file +10
-0+{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif
It's the same than for the function tracer here, ftrace_graph_caller must save the scratch registers to the stack.
Documentation/trace/ftrace-implementation.txt line 188
Quoted 10 lines
Documentation/trace/ftrace-implementation.txt new file +10
-0+ +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Here you are returning from a function, so you might be able to skimp +on things saved/restored.
It would be nice to add details about that, especially about a constant rule:return_to_handler must save/restore the return value of the current exiting function around ftrace_return_to_handler call.
And this return value might be stored in more than one register for 64 bits return values.
But we don't need to save/restore the other scratch registers because the traced function is exiting and won't need anymore values stored in them.
Also, we had some problems with return_to_handler in x86-64. We needed to allocate a large stack room (0x80 bytes) before calling ftrace_return_to_handler(). The funny thing is that we still don't know why we needed to do that, but omitting that resulted in crashes :-)
Quoted 18 lines
Documentation/trace/ftrace-implementation.txt new file +18
-0+#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point(); +} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- +
Documentation/trace/ftrace-implementation.txt line 216
Quoted 10 lines
Documentation/trace/ftrace-implementation.txt new file +10
-0+If you can't trace NMI functions, then skip this option. + +<details to be filled> + + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled> +
This part doesn't need more for now because it may change soon since the syscall tracing is currently reworked. We'll fill it once it reaches a more established state.
Documentation/trace/ftrace-implementation.txt line 229
Quoted 12 lines
Documentation/trace/ftrace-implementation.txt new file +12
-0+HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled> + + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>
But this part is important :)
Quoted 69 lines
Documentation/trace/ftrace.txt +6
-0@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such. + + The File System ---------------kernel/trace/Kconfig +13
-3@@ -11,31 +11,41 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_FTRACE_SYSCALLS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE boolSignature · 1 line
1.6.3.1
Other than what I commented, it looks very good.
Thanks!
Frederic.
Re: [PATCH v3] ftrace: document basic ftracer/ftracer graph needs
Quoted 4 lines · Frederic Weisbecker
Quoted 1 line · Mike Frysinger
+HAVE_FUNCTION_GRAPH_TRACER
Should be
HAVE_FUNCTION_TRACER, right?
yes
Quoted 6 lines
There is one crucial missing thing, I mean "save all state needed by the ABI" can be more detailed.
do_trace_must_
save the scratch and argument registers to the stack because
the traced function may have parameters passed by registers, initialized things on scratch registers, and this state
must be left intact before callingftrace_trace_function()
parameters passed by registers, yes, but as for scratch registers, that depends on the toolchain and where the mcount invocation occurs. if it's before the function prolog, then no, it doesnt need to worry about any scratch registers. if it's after, then yes, it probably needs to worry about those things. but this is why i have a paragraph saying "go read your abi documentation" and review glibc.
Quoted 14 lines
Documentation/trace/ftrace-implementation.txt line 188
Quoted 9 lines
Documentation/trace/ftrace-implementation.txt new file +9
-0+For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Here you are returning from a function, so you might be able to skimp +on things saved/restored.It would be nice to add details about that, especially about a constant rule:
return_to_handlermust save/restore the return value of the current exiting function aroundftrace_return_to_handlercall.
Quoted 5 lines
And this return value might be stored in more than one register for 64 bits return values.
But we don't need to save/restore the other scratch registers because the traced function is exiting and won't need anymore values stored in them.
i'm not familiar with other architectures and crazy shit that might go down here which is why i kind of skimped on details. for the Blackfin port, i know what i have to do -- just save/restore the return registers (r0 for 32bits, +r1 for 64bits, +p0 for >64bits). but i purposefully tried to avoid ABI details because i dont want this turning into "on <arch>, do <whole bunch of details>, on <arch2>, do <whole bunch of details>, ....".
the scratch register is more because the exit code is coming after the function epilog rather than "exiting it" ...
i dont mind adding tips, but the last thing i want is people complaining that they did what the docs said and now things crashed because they didnt fully grasp the "it's your ABI, so it's your problem".
Quoted 4 lines
Also, we had some problems with
return_to_handlerin x86-64. We needed to allocate a large stack room (0x80bytes) before callingftrace_return_to_handler(). The funny thing is that we still don't know why we needed to do that, but omitting that resulted in crashes :-)
without knowing anything about x86-64 treating of the stack, it does seem weird. with the Blackfin arch, the called function is responsible for allocating its own space.
Quoted 9 lines
Documentation/trace/ftrace-implementation.txt line 215
Quoted 4 lines
Documentation/trace/ftrace-implementation.txt new file +4
-0+HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled>This part doesn't need more for now because it may change soon since the syscall tracing is currently reworked. We'll fill it once it reaches a more established state.
Quoted 6 lines
Documentation/trace/ftrace-implementation.txt line 229
Quoted 4 lines
Documentation/trace/ftrace-implementation.txt new file +4
-0+HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>But this part is important :)
i filled in what i could reverse engineer ... and these two bits looked way more complicated than was worth me trying to figure out. these are what i'd be interested in next though.
-mike
Re: [PATCH v3] ftrace: document basic ftracer/ftracer graph needs
Quoted 15 lines · Mike Frysinger
Quoted 4 lines · Frederic Weisbecker
Quoted 1 line · Mike Frysinger
+HAVE_FUNCTION_GRAPH_TRACER
Should be
HAVE_FUNCTION_TRACER, right?yes
Quoted 6 lines
There is one crucial missing thing, I mean "save all state needed by the ABI" can be more detailed.
do_trace_must_
save the scratch and argument registers to the stack because
the traced function may have parameters passed by registers, initialized things on scratch registers, and this state
must be left intact before callingftrace_trace_function()
Quoted 6 lines
parameters passed by registers, yes, but as for scratch registers, that depends on the toolchain and where the mcount invocation occurs. if it's before the function prolog, then no, it doesnt need to worry about any scratch registers. if it's after, then yes, it probably needs to worry about those things. but this is why i have a paragraph saying "go read your abi documentation" and review glibc.
Ok, then in this case, just talk about it as a constraint that may be or not handled by the arch specific mcount.
It's really a prerequisite to check on every archs so I think it's worth showing this mandatory stage in the recipe.
Quoted 20 lines
Quoted 14 lines
Documentation/trace/ftrace-implementation.txt line 188
Quoted 9 lines
Documentation/trace/ftrace-implementation.txt new file +9
-0+For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Here you are returning from a function, so you might be able to skimp +on things saved/restored.It would be nice to add details about that, especially about a constant rule:
return_to_handlermust save/restore the return value of the current exiting function aroundftrace_return_to_handlercall.Quoted 5 lines
And this return value might be stored in more than one register for 64 bits return values.
But we don't need to save/restore the other scratch registers because the traced function is exiting and won't need anymore values stored in them.
Quoted 7 lines
i'm not familiar with other architectures and crazy shit that might go down here which is why i kind of skimped on details. for the Blackfin port, i know what i have to do -- just save/restore the return registers (r0 for 32bits, +r1 for 64bits, +p0 for >64bits). but i purposefully tried to avoid ABI details because i dont want this turning into "on <arch>, do <whole bunch of details>, on <arch2>, do <whole bunch of details>, ....".
That's not what I'm suggesting.
As I explained above, it's about a generic and cross arch constraint.
The details of implementation are only a concern for the developer.
Quoted 3 lines
the scratch register is more because the exit code is coming after the function epilog rather than "exiting it" ...
Yeah, that's what I meant by "exiting" :)
Quoted 5 lines
i dont mind adding tips, but the last thing i want is people complaining that they did what the docs said and now things crashed because they didnt fully grasp the "it's your ABI, so it's your problem".
I'm only talking about generic rules here. Every ABI has conventions about calls, scratch registers, etc...
But here we are using tracing code in very fragile paths that have common constraints on every archs. Those constraints concern common properties of every CPU, the implementation which implement these constraints is only a concern of the arch developer.
Quoted 8 lines
Quoted 4 lines
Also, we had some problems with
return_to_handlerin x86-64. We needed to allocate a large stack room (0x80bytes) before callingftrace_return_to_handler(). The funny thing is that we still don't know why we needed to do that, but omitting that resulted in crashes :-)without knowing anything about x86-64 treating of the stack, it does seem weird. with the Blackfin arch, the called function is responsible for allocating its own space.
That's also what does x86, really that's a mystery..
Quoted 16 lines
Quoted 9 lines
Documentation/trace/ftrace-implementation.txt line 215
Quoted 4 lines
Documentation/trace/ftrace-implementation.txt new file +4
-0+HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled>This part doesn't need more for now because it may change soon since the syscall tracing is currently reworked. We'll fill it once it reaches a more established state.
Quoted 6 lines
Documentation/trace/ftrace-implementation.txt line 229
Quoted 4 lines
Documentation/trace/ftrace-implementation.txt new file +4
-0+HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>But this part is important :)
Quoted 4 lines
i filled in what i could reverse engineer ... and these two bits looked way more complicated than was worth me trying to figure out. these are what i'd be interested in next though.
-mike
Never mind, this documentation can be filled through several iterations. This patch already provides very useful explanations.
Thanks,
Frederic.
PATCH v4 1/1 ftrace: document basic ftracer/ftracer graph needs
In reply to Frederic WeisbeckerWhile implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
---
v4
- fix HAVE_FUNCTION_TRACER typo in header
- add some more details about ABI state around mcount/return_to_handler
Documentation/trace/ftrace-implementation.txt | 233 +++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 252 insertions(+), 3 deletions(-) create mode 100644 Documentation/trace/ftrace-implementation.txt
Documentation/trace/ftrace-implementation.txt new file +233 -0
@@ -0,0 +1,233 @@ + ftrace guts + =========== + +Introduction +------------ + +Here we will cover the architecture pieces that the common ftrace code relies +on for proper functioning. Things are broken down into increasing complexity +so that you can start at the top and get at least basic functionality. + +Note that this focuses on architecture implementation details only. If you +want more explanation of a feature in terms of common code, review the common +ftrace.txt file. + + +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h + + +HAVE_FUNCTION_TRACER +-------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'main(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount +We'll make the assumption below that the symbol is "mcount" just to keep things +nice and simple in the examples. + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage (argument/scratch/etc...) +is a major issue at this point, especially in relation to the location of the +mcount call (before/after function prologue). You might also want to look at +how glibc has implemented the mcount function for your architecture. It might +be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; + +do_trace: + + /* save all state needed by the ABI (see paragraph above) */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +} + +Don't forget to export mcount for modules ! +extern void mcount(void); +EXPORT_SYMBOL(mcount); + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address. + +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif + +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Since you are returning from a function (after the epilogue), you might +be able to skimp on things saved/restored (usually just registers used to pass +return values). + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI (see paragraph above) */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point(); +} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- + +If you can't trace NMI functions, then skip this option. + +<details to be filled> + + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled> + + +HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled> + + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>
Documentation/trace/ftrace.txt +6 -0
@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such. + + The File System ---------------
kernel/trace/Kconfig +13 -3
@@ -11,31 +11,41 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_FTRACE_SYSCALLS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE bool
Signature · 1 line
1.6.3.1
Re: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 325 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
Signed-off-by Mike Frysinger <[email protected]>---
v4
- fixHAVE_FUNCTION_TRACERtypo in header
- add some more details about ABI state around mcount/return_to_handlerDocumentation/trace/ftrace-implementation.txt | 233 +++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 252 insertions(+), 3 deletions(-) create mode 100644 Documentation/trace/ftrace-implementation.txtDocumentation/trace/ftrace-implementation.txt new file +233
-0@@ -0,0 +1,233 @@ + ftrace guts + =========== + +Introduction +------------ + +Here we will cover the architecture pieces that the common ftrace code relies +on for proper functioning. Things are broken down into increasing complexity +so that you can start at the top and get at least basic functionality. + +Note that this focuses on architecture implementation details only. If you +want more explanation of a feature in terms of common code, review the common +ftrace.txt file. + + +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h + + +HAVE_FUNCTION_TRACER +-------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'main(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount +We'll make the assumption below that the symbol is "mcount" just to keep things +nice and simple in the examples. + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage (argument/scratch/etc...) +is a major issue at this point, especially in relation to the location of the +mcount call (before/after function prologue). You might also want to look at +how glibc has implemented the mcount function for your architecture. It might +be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; + +do_trace: + + /* save all state needed by the ABI (see paragraph above) */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +} + +Don't forget to export mcount for modules ! +extern void mcount(void); +EXPORT_SYMBOL(mcount); + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address. + +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif + +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Since you are returning from a function (after the epilogue), you might +be able to skimp on things saved/restored (usually just registers used to pass +return values). + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI (see paragraph above) */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point(); +} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- + +If you can't trace NMI functions, then skip this option. + +<details to be filled> + + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled> + + +HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled> + + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>Documentation/trace/ftrace.txt +6
-0@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such. + + The File System ---------------kernel/trace/Kconfig +13
-3@@ -11,31 +11,41 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_FTRACE_SYSCALLS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE boolSignature · 1 line
1.6.3.1
Looks better now,
Thanks.
Re: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
In reply to Mike FrysingerQuoted 4 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
Steven: were you going to pick this up ? or should i forward to Andrew ? -mike
Re: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 7 lines · Mike Frysinger
Quoted 4 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
Steven: were you going to pick this up ? or should i forward to Andrew ?
The .32 development window for the tracing tree starts in about a week. We'll probably fast-track your documentation bits to mainline
- but until then we try to watch for regressions and make sure -rc1 is round.
Thanks,
Ingo
Re: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 7 lines · Ingo Molnar
Quoted 6 lines · Mike Frysinger
Quoted 4 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
Quoted 6 lines
Quoted 1 line
Steven: were you going to pick this up ? or should i forward to Andrew ?
The .32 development window for the tracing tree starts in about a week. We'll probably fast-track your documentation bits to mainline
- but until then we try to watch for regressions and make sure -rc1 is round.
unless i missed something, this wasnt picked up
-mike
Steven Rostedt
maintainerRe: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 8 lines · Mike Frysinger
Quoted 7 lines · Ingo Molnar
Quoted 6 lines · Mike Frysinger
Quoted 4 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Steven: were you going to pick this up ? or should i forward to Andrew ?
The .32 development window for the tracing tree starts in about a week. We'll probably fast-track your documentation bits to mainline
- but until then we try to watch for regressions and make sure -rc1 is round.unless i missed something, this wasnt picked up
Ah, somehow this got put in my "done" pile. I'll review it tomorrow and if there's no issue with it, I'll pull it in and pass it along.
Thanks,
-- Steve
Steven Rostedt
maintainerRe: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
In reply to Mike FrysingerSorry, I really should have reviewed this earlier, but it fell into the wrong mbox.
Documentation/trace/ftrace-implementation.txt
Quoted 18 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
Signed-off-by Mike Frysinger <[email protected]>---
v4
- fixHAVE_FUNCTION_TRACERtypo in header
- add some more details about ABI state around mcount/return_to_handlerDocumentation/trace/ftrace-implementation.txt | 233 +++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 252 insertions(+), 3 deletions(-) create mode 100644 Documentation/trace/ftrace-implementation.txtDocumentation/trace/ftrace-implementation.txt +0
-0
Can you rename this to function-tracer-design.txt
ftrace != function tracer, although we pushed it that way in the beginning. But now it is a much broader scope of tracing, and we don't want to confuse people anymore. This is a function tracing design document, and does not get into other aspects of ftrace.
Documentation/trace/ftrace-implementation.txt line 1
Quoted 6 lines
new file mode 100644
index 0000000..6dbd5afDocumentation/trace/ftrace-implementation.txt new file +1
-0@@ -0,0 +1,233 @@ + ftrace guts
s/ftrace/function tracer/
Documentation/trace/ftrace-implementation.txt line 7
Quoted 6 lines
Documentation/trace/ftrace-implementation.txt new file +6
-0+ =========== + +Introduction +------------ + +Here we will cover the architecture pieces that the common ftrace code relies
s/ftrace/function tracing/
Quoted 4 lines
Documentation/trace/ftrace-implementation.txt new file +4
-0+on for proper functioning. Things are broken down into increasing complexity +so that you can start at the top and get at least basic functionality. + +Note that this focuses on architecture implementation details only. If you
Documentation/trace/ftrace-implementation.txt line 21
Quoted 10 lines
Documentation/trace/ftrace-implementation.txt new file +10
-0+want more explanation of a feature in terms of common code, review the common +ftrace.txt file. + + +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h
FYI but you can include it too. We force the use of TRACE_IRQFLAGS and STACKTRACE because we don't want to write a bunch of exceptions to implement some nice options of ftrace (and yes it's ftrace this time). You can see that PPC32 is the exception for IRQFLAGS because it has issues with tracing interrupts off in assembly. Once that is solved, it will have it enabled again.
Documentation/trace/ftrace-implementation.txt line 33
Quoted 12 lines
Documentation/trace/ftrace-implementation.txt new file +12
-0+ + +HAVE_FUNCTION_TRACER +-------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'main(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount
nice!
Quoted 170 lines
Documentation/trace/ftrace-implementation.txt new file +170
-0+We'll make the assumption below that the symbol is "mcount" just to keep things +nice and simple in the examples. + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage (argument/scratch/etc...) +is a major issue at this point, especially in relation to the location of the +mcount call (before/after function prologue). You might also want to look at +how glibc has implemented the mcount function for your architecture. It might +be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; + +do_trace: + + /* save all state needed by the ABI (see paragraph above) */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +} + +Don't forget to export mcount for modules ! +extern void mcount(void); +EXPORT_SYMBOL(mcount); + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address. + +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif + +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Since you are returning from a function (after the epilogue), you might +be able to skimp on things saved/restored (usually just registers used to pass +return values). + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI (see paragraph above) */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point();
Documentation/trace/ftrace-implementation.txt line 213
Quoted 10 lines
Documentation/trace/ftrace-implementation.txt new file +10
-0+} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- + +If you can't trace NMI functions, then skip this option. + +<details to be filled>
I guess I could fill this out later.
Documentation/trace/ftrace-implementation.txt line 219
Quoted 6 lines
Documentation/trace/ftrace-implementation.txt new file +6
-0+ + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled>
This would be for Frederic.
Documentation/trace/ftrace-implementation.txt line 227
Quoted 8 lines
Documentation/trace/ftrace-implementation.txt new file +8
-0+ + +HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled>
I could do this too.
Documentation/trace/ftrace-implementation.txt line 233
Quoted 6 lines
Documentation/trace/ftrace-implementation.txt new file +6
-0+ + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>
This is the fun part. I'll write up something (need to do it anyway). I'll try to get to it when I'm back from Dresden in October.
Documentation/trace/ftrace.txt disabled, and more (ftrace allows for tracer plugins, which · line 33
Quoted 12 lines
Documentation/trace/ftrace.txt +4
-0@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such.
Might want to specify that this is for the function tracer.
Quoted 44 lines
Documentation/trace/ftrace.txt +2
-0+ + The File System ---------------kernel/trace/Kconfig +11
-3@@ -11,31 +11,41 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt
kernel/trace/Kconfig config NOP_TRACER · line 51
Quoted 10 lines
kernel/trace/Kconfig +2
-0config HAVE_HW_BRANCH_TRACER bool config HAVE_FTRACE_SYSCALLS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE bool
Other than this, it looks good.
Could you make a v5, and send it to me. I promise that I wont slip it in the done folder again ;-)
-- Steve
Re: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 9 lines · Steven Rostedt
Quoted 8 lines · Mike Frysinger
Documentation/trace/ftrace.txt +0
-0@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, whichmeans that the list of tracers can always grow).
Quoted 6 lines
Documentation/trace/ftrace.txt disabled, and more (ftrace allows for tracer plugins, which · line 33
Quoted 4 lines
Documentation/trace/ftrace.txt +4
-0+Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such.Might want to specify that this is for the function tracer.
well, all of ftrace.txt says "function tracer", so if/when someone cleans that up, this will get fixed in the process.
Quoted 45 lines
Quoted 45 lines
kernel/trace/Kconfig +0
-0@@ -11,31 +11,41 @@ config NOP_TRACERconfig
HAVE_FTRACE_NMI_ENTER
bool
+ help
+ SeeDocumentation/trace/ftrace-implementation.txtconfig
HAVE_FUNCTION_TRACER
bool
+ help
+ SeeDocumentation/trace/ftrace-implementation.txtconfig
HAVE_FUNCTION_GRAPH_TRACER
bool
+ help
+ SeeDocumentation/trace/ftrace-implementation.txtconfig
HAVE_FUNCTION_TRACE_MCOUNT_TEST
bool
help
- This gets selected when the arch tests thefunction_trace_stop
- variable at the mcount call site. Otherwise, this variable
- is tested by the called function.
+ SeeDocumentation/trace/ftrace-implementation.txtconfig
HAVE_DYNAMIC_FTRACE
bool
+ help
+ SeeDocumentation/trace/ftrace-implementation.txtconfig
HAVE_FTRACE_MCOUNT_RECORD
bool
+ help
+ SeeDocumentation/trace/ftrace-implementation.txtconfig
HAVE_HW_BRANCH_TRACER
boolconfig
HAVE_FTRACE_SYSCALLS
bool
+ help
+ SeeDocumentation/trace/ftrace-implementation.txt
Quoted 4 lines
Quoted 2 lines
config
TRACER_MAX_TRACE
boolOther than this, it looks good.
so you want me to drop the Kconfig pieces ? there was discussion in this thread about why the Kconfig pieces were here ... -mike
Steven Rostedt
maintainerRe: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 17 lines · Mike Frysinger
Quoted 9 lines · Steven Rostedt
Quoted 8 lines · Mike Frysinger
Documentation/trace/ftrace.txt +0
-0@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow).Quoted 6 lines
Documentation/trace/ftrace.txt disabled, and more (ftrace allows for tracer plugins, which · line 33
Quoted 4 lines
Documentation/trace/ftrace.txt +4
-0+Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such.Might want to specify that this is for the function tracer.
Quoted 2 lines
well, all of ftrace.txt says "function tracer", so if/when someone cleans that up, this will get fixed in the process.
Well things do need to be clean up, the original code had the function tracer called ftrace, which it no longer does. Why add more incorrect documentation just to match the previous incorrect documentation?
Quoted 46 lines
Quoted 46 lines
Quoted 46 lines
kernel/trace/Kconfig +13
-3@@ -11,31 +11,41 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_FTRACE_SYSCALLS bool + help + See Documentation/trace/ftrace-implementation.txt
Quoted 7 lines
Quoted 4 lines
kernel/trace/Kconfig config NOP_TRACER · line 51
Quoted 2 lines
kernel/trace/Kconfig +0
-0config TRACER_MAX_TRACE boolOther than this, it looks good.
so you want me to drop the
Kconfigpieces ? there was discussion in this thread about why theKconfigpieces were here ...
I'm fine with the Kconfig, I was referencing my own comments, not the above.
-- Steve
Re: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 18 lines · Steven Rostedt
Quoted 17 lines · Mike Frysinger
Quoted 15 lines · Steven Rostedt
Documentation/trace/ftrace.txt disabled, and more (ftrace allows for tracer plugins, which · line 33
Quoted 12 lines · Mike Frysinger
Documentation/trace/ftrace.txt +0
-0@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, whichmeans that the list of tracers can always grow).
Documentation/trace/ftrace.txt +4
-0+Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such.Might want to specify that this is for the function tracer.
Quoted 6 lines
Quoted 2 lines
well, all of ftrace.txt says "function tracer", so if/when someone cleans that up, this will get fixed in the process.
Well things do need to be clean up, the original code had the function tracer called ftrace, which it no longer does. Why add more incorrect documentation just to match the previous incorrect documentation?
if the filename is updated, do i really need to duplicate it ?
See function-tracer-design.txt for details for arch porters and such.
if people add other design/porting info, this can be a simple list and the filename itself can imply the details contained within ... -mike
Steven Rostedt
maintainerRe: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 26 lines · Mike Frysinger
Quoted 18 lines · Steven Rostedt
Quoted 17 lines · Mike Frysinger
Quoted 15 lines · Steven Rostedt
Documentation/trace/ftrace.txt disabled, and more (ftrace allows for tracer plugins, which · line 33
Quoted 12 lines · Mike Frysinger
Documentation/trace/ftrace.txt +4
-0@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such.Might want to specify that this is for the function tracer.
Quoted 6 lines
Quoted 2 lines
well, all of ftrace.txt says "function tracer", so if/when someone cleans that up, this will get fixed in the process.
Well things do need to be clean up, the original code had the function tracer called ftrace, which it no longer does. Why add more incorrect documentation just to match the previous incorrect documentation?
Quoted 6 lines
if the filename is updated, do i really need to duplicate it ?
See function-tracer-design.txt for details for arch porters and such.
if people add other design/porting info, this can be a simple list and the filename itself can imply the details contained within ...
Fine, then rename it to ftrace-design.txt (this corresponds to other docs in Documentation, like lockdep-design.txt and rt-mutex-design.txt)
Then in the file, start it with "Function tracer implementation" or something similar.
Re: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 31 lines · Steven Rostedt
Quoted 30 lines · Mike Frysinger
Quoted 18 lines · Steven Rostedt
Quoted 17 lines · Mike Frysinger
Quoted 15 lines · Steven Rostedt
Documentation/trace/ftrace.txt disabled, and more (ftrace allows for tracer plugins, which · line 33
Quoted 12 lines · Mike Frysinger
Documentation/trace/ftrace.txt +0
-0@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, whichmeans that the list of tracers can always grow).
Documentation/trace/ftrace.txt +4
-0+Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such.Might want to specify that this is for the function tracer.
Quoted 6 lines
Quoted 2 lines
well, all of ftrace.txt says "function tracer", so if/when someone cleans that up, this will get fixed in the process.
Well things do need to be clean up, the original code had the function tracer called ftrace, which it no longer does. Why add more incorrect documentation just to match the previous incorrect documentation?
if the filename is updated, do i really need to duplicate it ?
See function-tracer-design.txt for details for arch porters and such.
Quoted 8 lines
Quoted 2 lines
if people add other design/porting info, this can be a simple list and the filename itself can imply the details contained within ...
Fine, then rename it to ftrace-design.txt (this corresponds to other docs in Documentation, like lockdep-design.txt and rt-mutex-design.txt)
Then in the file, start it with "Function tracer implementation" or something similar.
i'm fine with either way, i was just trying to point out what i thought was redundant information in the brief description. "See function-tracer-design.txt for details on the function tracer design.".
although, if we're going to add nmi/syscall/dynamic tracer to the same file, it probably makes sense to go with "ftrace-design.txt". -mike
Steven Rostedt
maintainerRe: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
Quoted 2 lines · Mike Frysinger
although, if we're going to add nmi/syscall/dynamic tracer to the same file, it probably makes sense to go with "ftrace-design.txt".
Lets go with ftrace-design.txt.
Thanks,
-- Steve
Re: [PATCH v4] ftrace: document basic ftracer/ftracer graph needs
In reply to Steven RostedtQuoted 6 lines · Steven Rostedt
Documentation/trace/ftrace-implementation.txt line 219
Quoted 4 lines
Documentation/trace/ftrace-implementation.txt new file +4
-0+HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled>This would be for Frederic.
Yep. Especially there may be some movement from arch code to core wrt syscall tracing in the .33 development cycle so once it is done I can document this part.
PATCH v5 1/1 ftrace: document basic ftracer/ftracer graph needs
In reply to Steven RostedtWhile implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
--- Documentation/trace/ftrace-design.txt | 233 +++++++++++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 252 insertions(+), 3 deletions(-) create mode 100644 Documentation/trace/ftrace-design.txt
Documentation/trace/ftrace-design.txt new file +233 -0
@@ -0,0 +1,233 @@ + function tracer guts + ==================== + +Introduction +------------ + +Here we will cover the architecture pieces that the common function tracing +code relies on for proper functioning. Things are broken down into increasing +complexity so that you can start simple and at least get basic functionality. + +Note that this focuses on architecture implementation details only. If you +want more explanation of a feature in terms of common code, review the common +ftrace.txt file. + + +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h + + +HAVE_FUNCTION_TRACER +-------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'main(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount +We'll make the assumption below that the symbol is "mcount" just to keep things +nice and simple in the examples. + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage (argument/scratch/etc...) +is a major issue at this point, especially in relation to the location of the +mcount call (before/after function prologue). You might also want to look at +how glibc has implemented the mcount function for your architecture. It might +be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; + +do_trace: + + /* save all state needed by the ABI (see paragraph above) */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +} + +Don't forget to export mcount for modules ! +extern void mcount(void); +EXPORT_SYMBOL(mcount); + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address. + +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif + +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Since you are returning from a function (after the epilogue), you might +be able to skimp on things saved/restored (usually just registers used to pass +return values). + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI (see paragraph above) */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point(); +} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- + +If you can't trace NMI functions, then skip this option. + +<details to be filled> + + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled> + + +HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled> + + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>
Documentation/trace/ftrace.txt +6 -0
@@ -26,6 +26,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-design.txt for details for arch porters and such. + + The File System ---------------
kernel/trace/Kconfig +13 -3
@@ -11,12 +11,18 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_FP_TEST bool
@@ -28,21 +34,25 @@ config HAVE_FUNCTION_GRAPH_FP_TEST config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_SYSCALL_TRACEPOINTS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE bool
Signature · 1 line
1.6.4.2
Steven Rostedt
maintainer[PATCH] [GIT PULL] ftrace: document basic ftracer/ftracer graph needs
Ingo,
Please pull the latest tip/tracing/core2 tree, which can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
tip/tracing/core2
Mike Frysinger (1):
ftrace: document function and function graph implementation
----
Documentation/trace/ftrace-design.txt | 233 +++++++++++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 252 insertions(+), 3 deletions(-) --------------------------- commit 555f386c98cc93890f48fdea098936755270304b Author: Mike Frysinger <[email protected]> Date: Mon Sep 14 20:10:15 2009 -0400
ftrace: document function and function graph implementation
While implementing function tracer and function tracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
LKML-Reference: <[email protected]>
Documentation/trace/ftrace-design.txt new file +233 -0
@@ -0,0 +1,233 @@ + function tracer guts + ==================== + +Introduction +------------ + +Here we will cover the architecture pieces that the common function tracing +code relies on for proper functioning. Things are broken down into increasing +complexity so that you can start simple and at least get basic functionality. + +Note that this focuses on architecture implementation details only. If you +want more explanation of a feature in terms of common code, review the common +ftrace.txt file. + + +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h + + +HAVE_FUNCTION_TRACER +-------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'main(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount +We'll make the assumption below that the symbol is "mcount" just to keep things +nice and simple in the examples. + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage (argument/scratch/etc...) +is a major issue at this point, especially in relation to the location of the +mcount call (before/after function prologue). You might also want to look at +how glibc has implemented the mcount function for your architecture. It might +be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; + +do_trace: + + /* save all state needed by the ABI (see paragraph above) */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +} + +Don't forget to export mcount for modules ! +extern void mcount(void); +EXPORT_SYMBOL(mcount); + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address. + +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif + +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Since you are returning from a function (after the epilogue), you might +be able to skimp on things saved/restored (usually just registers used to pass +return values). + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI (see paragraph above) */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point(); +} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- + +If you can't trace NMI functions, then skip this option. + +<details to be filled> + + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled> + + +HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled> + + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>
Documentation/trace/ftrace.txt +6 -0
@@ -26,6 +26,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-design.txt for details for arch porters and such. + + The File System ---------------
kernel/trace/Kconfig +13 -3
@@ -11,12 +11,18 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_FP_TEST bool
@@ -28,21 +34,25 @@ config HAVE_FUNCTION_GRAPH_FP_TEST config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_SYSCALL_TRACEPOINTS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE bool
[tip:tracing/core] ftrace: document function and function graph implementation
In reply to Mike FrysingerCommit-ID: 555f386c98cc93890f48fdea098936755270304b Gitweb: http://git.kernel.org/tip/555f386c98cc93890f48fdea098936755270304b Author: Mike Frysinger <[email protected]> AuthorDate: Mon, 14 Sep 2009 20:10:15 -0400 Committer: Steven Rostedt <[email protected]> CommitDate: Mon, 14 Sep 2009 21:43:15 -0400
ftrace: document function and function graph implementation
While implementing function tracer and function tracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
LKML-Reference: <[email protected]>
--- Documentation/trace/ftrace-design.txt | 233 +++++++++++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 252 insertions(+), 3 deletions(-)
Documentation/trace/ftrace-design.txt new file +233 -0
@@ -0,0 +1,233 @@ + function tracer guts + ==================== + +Introduction +------------ + +Here we will cover the architecture pieces that the common function tracing +code relies on for proper functioning. Things are broken down into increasing +complexity so that you can start simple and at least get basic functionality. + +Note that this focuses on architecture implementation details only. If you +want more explanation of a feature in terms of common code, review the common +ftrace.txt file. + + +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h + + +HAVE_FUNCTION_TRACER +-------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'main(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount +We'll make the assumption below that the symbol is "mcount" just to keep things +nice and simple in the examples. + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage (argument/scratch/etc...) +is a major issue at this point, especially in relation to the location of the +mcount call (before/after function prologue). You might also want to look at +how glibc has implemented the mcount function for your architecture. It might +be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; + +do_trace: + + /* save all state needed by the ABI (see paragraph above) */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +} + +Don't forget to export mcount for modules ! +extern void mcount(void); +EXPORT_SYMBOL(mcount); + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address. + +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif + +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. + +Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Since you are returning from a function (after the epilogue), you might +be able to skimp on things saved/restored (usually just registers used to pass +return values). + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI (see paragraph above) */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point(); +} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- + +If you can't trace NMI functions, then skip this option. + +<details to be filled> + + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled> + + +HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled> + + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>
Documentation/trace/ftrace.txt +6 -0
@@ -26,6 +26,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-design.txt for details for arch porters and such. + + The File System ---------------
kernel/trace/Kconfig +13 -3
@@ -11,12 +11,18 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_FP_TEST bool
@@ -28,21 +34,25 @@ config HAVE_FUNCTION_GRAPH_FP_TEST config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_SYSCALL_TRACEPOINTS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE bool
Re: [PATCH 2/2 v2] ftrace: document basic ftracer/ftracer graph needs
In reply to Mike FrysingerQuoted 194 lines · Mike Frysinger
While implementing ftracer and ftracer graph support, I found the exact arch implementation details to be a bit lacking (and my x86 foo ain't great). So after pounding out support for the Blackfin arch, start documenting the requirements/details.
Signed-off-by Mike Frysinger <[email protected]>---
v2
- move toDocumentation/and add a lot more detailsnote: yes, i think every kconfig option that is documented should point to the relevant text file as this helps in the case of people who use kconfig's search function to look for things
Documentation/trace/ftrace-implementation.txt | 223 +++++++++++++++++++++++++ Documentation/trace/ftrace.txt | 6 + kernel/trace/Kconfig | 16 ++- 3 files changed, 242 insertions(+), 3 deletions(-) create mode 100644 Documentation/trace/ftrace-implementation.txtDocumentation/trace/ftrace-implementation.txt new file +168
-0@@ -0,0 +1,223 @@ + ftrace guts + =========== + +Introduction +------------ + +Here we will cover the architecture pieces that the common ftrace code relies +on for proper functioning. Things are broken down into increasing complexity +so that you can start at the top and get at least basic functionality. + +Note that this focuses on architecture implementation details only. If you +want more explanation of a feature in terms of common code, review the common +ftrace.txt file. + + +Prerequisites +------------- + +Ftrace relies on these features being implemented: + STACKTRACE_SUPPORT - implement save_stack_trace() + TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +You will need to implement the mcount and the ftrace_stub functions. + +The exact mcount symbol name will depend on your toolchain. Some call it +"mcount", "_mcount", or even "__mcount". You can probably figure it out by +running something like: + $ echo 'm(){}' | gcc -x c -S -o - - -pg | grep mcount + call mcount + +Keep in mind that the ABI that is in effect inside of the mcount function is +*highly* architecture/toolchain specific. We cannot help you in this regard, +sorry. Dig up some old documentation and/or find someone more familiar than +you to bang ideas off of. Typically, register usage/clobbering is a major +problem here. You might also want to look at how glibc has implemented the +mcount function for your architecture. It might be (semi-)relevant. + +The mcount function should check the function pointer ftrace_trace_function +to see if it is set to ftrace_stub. If it is, there is nothing for you to do, +so return immediately. If it isn't, then call that function in the same way +the mcount function normally calls __mcount_internal -- the first argument is +the "frompc" while the second argument is the "selfpc" (adjusted to remove the +size of the mcount call that is embedded in the function). + +For example, if the function foo() calls bar(), when the bar() function calls +mcount(), the arguments mcount() will pass to the tracer are: + "frompc" - the address bar() will use to return to foo() + "selfpc" - the address bar() (with _mcount() size adjustment) + +Also keep in mind that this mcount function will be called *a lot*, so +optimizing for the default case of no tracer will help the smooth running of +your system when tracing is disabled. So the start of the mcount function is +typically the bare min with checking things before returning. That also means +the code flow should usually kept linear (i.e. no branching in the nop case). +This is of course an optimization and not a hard requirement. + +Here is some pseudo code that should help (these functions should actually be +implemented in assembly): + +void ftrace_stub(void) +{ + return; +} + +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + + /* restore any bare state */ + + return; + +do_trace: + + /* save all state needed by the ABI */ + + unsigned long frompc = ...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + ftrace_trace_function(frompc, selfpc); + + /* restore all state needed by the ABI */ +} + + +HAVE_FUNCTION_TRACE_MCOUNT_TEST +------------------------------- + +This is an optional optimization for the normal case when tracing is turned off +in the system. If you do not enable this Kconfig option, the common ftrace +code will take care of doing the checking for you. + +To support this feature, you only need to check the function_trace_stop +variable in the mcount function. If it is non-zero, there is no tracing to be +done at all, so you can return. + +This additional pseudo code would simply be: +void mcount(void) +{ + /* save any bare state needed in order to do initial checking */ + ++ if (function_trace_stop) ++ return; + + extern void (*ftrace_trace_function)(unsigned long, unsigned long); + if (ftrace_trace_function != ftrace_stub) +... + + +HAVE_FUNCTION_GRAPH_TRACER +-------------------------- + +Deep breath ... time to do some real work. Here you will need to update the +mcount function to check ftrace graph function pointers, as well as implement +some functions to save (hijack) and restore the return address. + +The mcount function should check the function pointers ftrace_graph_return +(compare to ftrace_stub) and ftrace_graph_entry (compare to +ftrace_graph_entry_stub). If either of those are not set to the relevant stub +function, call the arch-specific function ftrace_graph_caller which in turn +calls the arch-specific function prepare_ftrace_return. Neither of these +function names are strictly required, but you should use them anyways to stay +consistent across the architecture ports -- easier to compare & contrast +things. + +The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. + +Here is the updated mcount pseudo code: +void mcount(void) +{ +... + if (ftrace_trace_function != ftrace_stub) + goto do_trace; + ++#ifdef CONFIG_FUNCTION_GRAPH_TRACER ++ extern void (*ftrace_graph_return)(...); ++ extern void (*ftrace_graph_entry)(...); ++ if (ftrace_graph_return != ftrace_stub || ++ ftrace_graph_entry != ftrace_graph_entry_stub) ++ ftrace_graph_caller(); ++#endif + + /* restore any bare state */ +... + +Here is the pseudo code for the new ftrace_graph_caller assembly function: +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc);
Documentation/trace/ftrace-implementation.txt line 178
Quoted 10 lines
Documentation/trace/ftrace-implementation.txt new file +10
-0+ + /* restore all state needed by the ABI */ +} +#endif + +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. +
In reality, the prepare_ftrace_return() can have the same arguments as ftrace_trace_function(), and the fault recovery table is not needed.
in my mips-specific implementation(in the stage of -v2, i will send -v3 later), i removed the fault recovery table directly and implemented it in the following method:
as i know, prepare_ftrace_return() is used to check whether the calling function expect to trace, if yes, return the 'hook' function &return_to_handler, otherwise, return back to the parent_ip directly. so, here, i think there is no need to transfer the data via address, but just using the same arguments like ftrace_trace_function does.
unsigned longprepare_ftrace_return(unsgined long ip, unsigned long parent_ip)
{
[...]
if (ftrace_push_return_trace(parent_ip, ip, &trace.depth) == -EBUSY)
return parent_ip;
if (ftrace_graph_entry(&trace))
return (unsigned long)&return_to_handler;
[...]
return parent_ip;
}if using the above method, the fault recovery table is not needed again.
am i wrong? if this method is okay, so, this document should not only refer to the x86's implementation detail, but focus on the basic principle of prepare_ftrace_return().
thanks!
Wu Zhangjin
Quoted 111 lines
Documentation/trace/ftrace-implementation.txt new file +45
-0+Here is the pseudo code for the new return_to_handler assembly function. Note +that the ABI that applies here is different from what applies to the mcount +code. Here you are returning from a function, so you might be able to skimp +on things saved/restored. +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void return_to_handler(void) +{ + /* save all state needed by the ABI */ + + void (*original_return_point)(void) = ftrace_return_to_handler(); + + /* restore all state needed by the ABI */ + + /* this is usually either a return or a jump */ + original_return_point(); +} +#endif + + +HAVE_FTRACE_NMI_ENTER +--------------------- + +If you can't trace NMI functions, then skip this option. + +<details to be filled> + + +HAVE_FTRACE_SYSCALLS +--------------------- + +<details to be filled> + + +HAVE_FTRACE_MCOUNT_RECORD +------------------------- + +See scripts/recordmcount.pl for more info. + +<details to be filled> + + +HAVE_DYNAMIC_FTRACE +--------------------- + +<details to be filled>Documentation/trace/ftrace.txt +6
-0@@ -27,6 +27,12 @@ disabled, and more (ftrace allows for tracer plugins, which means that the list of tracers can always grow). +Implementation Details +---------------------- + +See ftrace-implementation.txt for details for arch porters and such. + + The File System ---------------kernel/trace/Kconfig +13
-3@@ -11,31 +11,41 @@ config NOP_TRACER config HAVE_FTRACE_NMI_ENTER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_GRAPH_TRACER bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FUNCTION_TRACE_MCOUNT_TEST bool help - This gets selected when the arch tests the function_trace_stop - variable at the mcount call site. Otherwise, this variable - is tested by the called function. + See Documentation/trace/ftrace-implementation.txt config HAVE_DYNAMIC_FTRACE bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_FTRACE_MCOUNT_RECORD bool + help + See Documentation/trace/ftrace-implementation.txt config HAVE_HW_BRANCH_TRACER bool config HAVE_FTRACE_SYSCALLS bool + help + See Documentation/trace/ftrace-implementation.txt config TRACER_MAX_TRACE bool
Re: [PATCH 2/2 v2] ftrace: document basic ftracer/ftracer graph needs
Quoted 22 lines · Wu Zhangjin
Quoted 21 lines · Mike Frysinger
Documentation/trace/ftrace-implementation.txt new file +20
-0+The arguments to prepare_ftrace_return are slightly different than what are +passed to ftrace_trace_function. The second argument "selfpc" is the same, +but the first argument should be a pointer to the "frompc". Typically this is +located on the stack. This allows the function to hijack the return address +temporarily to have it point to the arch-specific function return_to_handler. +That function will simply call the common ftrace_return_to_handler function and +that will return the original return address with which, you can return to the +original call site. +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void ftrace_graph_caller(void) +{ + /* save all state needed by the ABI */ + + unsigned long *frompc = &...; + unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; + prepare_ftrace_return(frompc, selfpc); + + /* restore all state needed by the ABI */ +} +#endif
Quoted 8 lines
Documentation/trace/ftrace-implementation.txt line 178
Quoted 6 lines
Documentation/trace/ftrace-implementation.txt new file +6
-0+ +For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. +and the fault recovery table is not needed.
i dont have one for the Blackfin port, but that's more because there is no fault recovery support in the Blackfin port ;)
Quoted 8 lines
In reality, the
prepare_ftrace_return()can have the same arguments asftrace_trace_function()
...
as i know,prepare_ftrace_return()is used to check whether the calling function expect to trace, if yes, return the 'hook' function&return_to_handler, otherwise, return back to theparent_ipdirectly. so, here, i think there is no need to transfer the data via address, but just using the same arguments likeftrace_trace_functiondoes.
hmm, that would make the implementation simpler, but i dont think you could do that if you implemented the fault handler. i cant really speak as to the requirement of the fault handler as i dont really know what/how it works -- i can only guess at they arent used in any way for Blackfin systems.
Quoted 10 lines
unsigned long
prepare_ftrace_return(unsgined long ip, unsigned long parent_ip){
[...]
if (ftrace_push_return_trace(parent_ip, ip, &trace.depth)==-EBUSY) returnparent_ip;if (
ftrace_graph_entry(&trace))
return (unsigned long)&return_to_handler;
Quoted 6 lines
[...]
return
parent_ip;
}if using the above method, the fault recovery table is not needed again.
and it would allow us to move this code into the common ftrace framework for people who dont do fault handlers.
Quoted 3 lines
am i wrong? if this method is okay, so, this document should not only refer to the x86's implementation detail, but focus on the basic principle of
prepare_ftrace_return().
ppc also uses the same method as x86 which is why i wrote it that way. -mike
Re: [PATCH 2/2 v2] ftrace: document basic ftracer/ftracer graph needs
hi, Mike
Quoted 38 lines
Quoted 7 lines
Documentation/trace/ftrace-implementation.txt line 178
Quoted 5 lines
Documentation/trace/ftrace-implementation.txt new file +5
-0+For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. +and the fault recovery table is not needed.
i dont have one for the Blackfin port, but that's more because there is no fault recovery support in the Blackfin port ;)
Quoted 8 lines
In reality, the
prepare_ftrace_return()can have the same arguments asftrace_trace_function()
...
as i know,prepare_ftrace_return()is used to check whether the calling function expect to trace, if yes, return the 'hook' function&return_to_handler, otherwise, return back to theparent_ipdirectly. so, here, i think there is no need to transfer the data via address, but just using the same arguments likeftrace_trace_functiondoes.hmm, that would make the implementation simpler, but i dont think you could do that if you implemented the fault handler. i cant really speak as to the requirement of the fault handler as i dont really know what/how it works -- i can only guess at they arent used in any way for Blackfin systems.
Quoted 12 lines
unsigned long
prepare_ftrace_return(unsgined long ip, unsigned long parent_ip){ [...] if (ftrace_push_return_trace(parent_ip, ip, &trace.depth) == -EBUSY) return parent_ip; if (ftrace_graph_entry(&trace)) return (unsigned long)&return_to_handler; [...]
Quoted 8 lines
Quoted 4 lines
return parent_ip; }if using the above method, the fault recovery table is not needed again.
and it would allow us to move this code into the common ftrace framework for people who dont do fault handlers.
Perhaps I miss something here? otherwise, this fault handler is not needed if not transfer the data via address and not modify the data in the place *address accordingly.
best regards,
Wu Zhangjin
Re: [PATCH 2/2 v2] ftrace: document basic ftracer/ftracer graph needs
Quoted 41 lines · Wu Zhangjin
Quoted 41 lines
Quoted 7 lines
Documentation/trace/ftrace-implementation.txt line 178
Quoted 5 lines
Documentation/trace/ftrace-implementation.txt new file +5
-0+For information on how to implement prepare_ftrace_return(), simply look at +the x86 version. The only architecture-specific piece in it is the setup of +the fault recovery table (the asm(...) code). The rest should be the same +across architectures. +and the fault recovery table is not needed.
i dont have one for the Blackfin port, but that's more because there is no fault recovery support in the Blackfin port ;)
Quoted 8 lines
In reality, the
prepare_ftrace_return()can have the same arguments asftrace_trace_function()
...
as i know,prepare_ftrace_return()is used to check whether the calling function expect to trace, if yes, return the 'hook' function&return_to_handler, otherwise, return back to theparent_ipdirectly. so, here, i think there is no need to transfer the data via address, but just using the same arguments likeftrace_trace_functiondoes.hmm, that would make the implementation simpler, but i dont think you could do that if you implemented the fault handler. i cant really speak as to the requirement of the fault handler as i dont really know what/how it works -- i can only guess at they arent used in any way for Blackfin systems.
Quoted 15 lines
unsigned long
prepare_ftrace_return(unsgined long ip, unsigned long parent_ip){
[...]
if (ftrace_push_return_trace(parent_ip, ip, &trace.depth)==-EBUSY) returnparent_ip;if (
ftrace_graph_entry(&trace))
return (unsigned long)&return_to_handler;[...]
return
parent_ip;
}
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
if using the above method, the fault recovery table is not needed again.
and it would allow us to move this code into the common ftrace framework for people who dont do fault handlers.
Perhaps I miss something here? otherwise, this fault handler is not needed if not transfer the data via address and not modify the data in the place *address accordingly.
this is based on my understanding of things, but really ive never worked with a mmu before, so ive never had to deal with fault handlers
...
in order for ftrace graph to work, the return address has to be modified to point to the arch-specific return_to_handler function. it doesnt matter where this modification occurs -- whether in the C prepare_ftrace_return() or in the arch assembly function calling prepare_ftrace_return().
the fault handler is there in case this modification fails for any reason. it doesnt matter if the code is restoring the old address in case one of the common ftrace functions returns an error (stack depth full/whatever).
-mike
Steven Rostedt
maintainerQuoted 4 lines · Mike Frysinger
The function graph tracer is called just "
function_graph" (no trailing "_tracer" needed).Signed-off-by Mike Frysinger <[email protected]>
Thanks, I'll queue it up.
-- Steve
Quoted 21 lines
--- Documentation/trace/ftrace.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)Documentation/trace/ftrace.txt +1
-1@@ -179,7 +179,7 @@ Here is the list of current tracers that may be configured. Function call tracer to trace all kernel functions. - "function_graph_tracer" + "function_graph" Similar to the function tracer except that the function tracer probes the functions on their entrySignature · 1 line
1.6.3.1
Steven Rostedt
maintainer[PATCH][GIT PULL] ftrace/documentation: fix typo in function grapher name
In reply to Mike FrysingerIngo,
Please pull the latest tip/tracing/ftrace tree, which can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
tip/tracing/ftrace
Mike Frysinger (1):
ftrace/documentation: fix typo in function grapher name
----
Documentation/trace/ftrace.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --------------------------- commit bc5c6c043d8381676339fb3da59cc4cc5921d368 Author: Mike Frysinger <[email protected]> Date: Wed Jun 10 04:48:41 2009 -0400
ftrace/documentation: fix typo in function grapher name
The function graph tracer is called just "function_graph" (no trailing "_tracer" needed).
LKML-Reference: <[email protected]>
Documentation/trace/ftrace.txt +1 -1
@@ -179,7 +179,7 @@ Here is the list of current tracers that may be configured. Function call tracer to trace all kernel functions. - "function_graph_tracer" + "function_graph" Similar to the function tracer except that the function tracer probes the functions on their entry
[tip:tracing/core] ftrace/documentation: fix typo in function grapher name
In reply to Mike FrysingerCommit-ID: bc5c6c043d8381676339fb3da59cc4cc5921d368 Gitweb: http://git.kernel.org/tip/bc5c6c043d8381676339fb3da59cc4cc5921d368 Author: Mike Frysinger <[email protected]> AuthorDate: Wed, 10 Jun 2009 04:48:41 -0400 Committer: Steven Rostedt <[email protected]> CommitDate: Wed, 10 Jun 2009 13:06:25 -0400
ftrace/documentation: fix typo in function grapher name
The function graph tracer is called just "function_graph" (no trailing "_tracer" needed).
LKML-Reference: <[email protected]>
--- Documentation/trace/ftrace.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
Documentation/trace/ftrace.txt +1 -1
@@ -179,7 +179,7 @@ Here is the list of current tracers that may be configured. Function call tracer to trace all kernel functions. - "function_graph_tracer" + "function_graph" Similar to the function tracer except that the function tracer probes the functions on their entry